hypso.utils.utils_file
Module Contents
Classes
Class used to display a progress bar on the screen for bulk downloading. |
Functions
|
Flatten a nested dictionary into a single level dictionary |
|
Convert nested dictionary to DataFrame. |
|
Compare two .nc files and returns a pandas DataFrame where each attribute, variable and values are compared side |
|
Calculate a colore accurate RGB image with HSI values. Original function used for the paper |
|
Find recursively all files in a specified directory. |
|
Find recursively a single file (or first occurance) in a specified directory. |
|
Find recursively a directory (or first occurance) in a specified directory. |
Check if a number is an integer |
|
|
Navigate recursively the structure of a .nc file and return the dictionary structure. The function will call |
|
Print the contents of a .nc file |
Converts 1D numpy array to string |
|
|
Navigate recursively the structure of a .nc file and print the structure. The function will call |
|
Find the closest Image lat and lon to the targat specificed that is also a water pixel. |
|
Calculate the great circle distance between two points on the earth (specified in decimal degrees). |
- hypso.utils.utils_file.flatten_dict(nested_dict)
Flatten a nested dictionary into a single level dictionary
- Parameters:
nested_dict (dict) – Nested dictionary to flatten
- Returns:
Single level dictionary with keys corresponding to the previous nested levels
- Return type:
dict
- hypso.utils.utils_file.nested_dict_to_df(values_dict)
Convert nested dictionary to DataFrame.
- Parameters:
values_dict (dict) – Nested dictionary
- Returns:
Dataframe where the nested level are displayed as a column.
- Return type:
pandas.DataFrame
- hypso.utils.utils_file.compare_netcdf_files(file1, file2)
Compare two .nc files and returns a pandas DataFrame where each attribute, variable and values are compared side by side.
- Parameters:
file1 (str) – Absolute path to file 1
file2 (str) – Absolute path to file 2
- Returns:
Dataframe with the compared characteristics
- Return type:
pandas.DataFrame
- hypso.utils.utils_file.HSI2RGB(wY, HSI, d=65, threshold=0.002)
Calculate a colore accurate RGB image with HSI values. Original function used for the paper “Creating RGB Images from Hyperspectral Images Using a Color Matching Function” by M. Magnusson, J. Sigurdsson, S. E. Armansson, M. O. Ulfarsson, H. Deborah and J. R. Sveinsson,
If you use this method, please cite the following paper: @INPROCEEDINGS{hsi2rgb, author={M. {Magnusson} and J. {Sigurdsson} and S. E. {Armansson} and M. O. {Ulfarsson} and H. {Deborah} and J. R. {Sveinsson}}, booktitle={IEEE International Geoscience and Remote Sensing Symposium}, title={Creating {RGB} Images from Hyperspectral Images using a Color Matching Function}, year={2020}, volume={}, number={}, pages={}}
Paper is available at https://www.researchgate.net/profile/Jakob_Sigurdsson
- Parameters:
wY – Wavelengths in nm
HSI – Hyperspectral Image as a (#pixels x #bands) matrix,
d (Literal[50, 55, 65, 75]) – Determines the illuminant used, if in doubt use d65. Values of 50, 55, 65, 75
threshold – True if RGB thesholding should be done to increase contrast
- Returns:
RGB Image array
- Return type:
numpy.ndarray
- class hypso.utils.utils_file.MyProgressBar(text_prefix)
Class used to display a progress bar on the screen for bulk downloading.
- __call__(block_num, block_size, total_size)
- hypso.utils.utils_file.find_all_files(path, str_in_file, suffix=None, type='partial')
Find recursively all files in a specified directory.
- Parameters:
path (pathlib.Path) – Directory path on which to look for files.
str_in_file (str) – Check if this string is in the file name
suffix (Union[str, None]) – Check if file extension matches the suffix provided
type (Literal[partial, exact]) – Type of comparisson of “str_in_file”. If “partial”, then “str_in_file” will be checked to be in the filename, otherwise “exact” will check if the filename matches exactly the string provided.
- Returns:
List of file paths
- Return type:
list
- hypso.utils.utils_file.find_file(path, str_in_file, suffix=None, type='partial')
Find recursively a single file (or first occurance) in a specified directory.
- Parameters:
path (pathlib.Path) – Directory path on which to look for files.
str_in_file (str) – Check if this string is in the file name
suffix (Union[str, None]) – Check if file extension matches the suffix provided
type (Literal[partial, exact]) – Type of comparisson of “str_in_file”. If “partial”, then “str_in_file” will be checked to be in the filename, otherwise “exact” will check if the filename matches exactly the string provided.
- Returns:
Path of the file or None if no file found
- Return type:
Union[pathlib.Path, None]
- hypso.utils.utils_file.find_dir(path, str_in_dir, type='partial')
Find recursively a directory (or first occurance) in a specified directory.
- Parameters:
path (pathlib.Path) – Directory path on which to look for files.
str_in_dir (str) – Check if this string is in the directory name
type (Literal[partial, exact]) – Type of comparisson of “str_in_file”. If “partial”, then “str_in_file” will be checked to be in the filename, otherwise “exact” will check if the filename matches exactly the string provided.
- Returns:
Path of the file or None if no file found
- Return type:
Union[pathlib.Path, None]
- hypso.utils.utils_file.is_integer_num(n)
Check if a number is an integer
- Parameters:
n – Number to check
- Returns:
Boolean value indicating whether or not the number is an integer
- Return type:
bool
Navigate recursively the structure of a .nc file and return the dictionary structure. The function will call itself to navigate recursively.
- Parameters:
nc_file (netCDF4.Dataset) – Open .nc Dataset file
path (str) – Relative path to append to recursivness
depth (int) – Depth at which we are navigating
- Returns:
Dictionary of dictionaries containing the structure of the .nc file
- Return type:
dict
- hypso.utils.utils_file.print_nc(nc_file_path)
Print the contents of a .nc file :param nc_file_path: Absolute path to a .nc file
- Returns:
No return
- Parameters:
nc_file_path (str) –
- Return type:
None
- hypso.utils.utils_file.list_array_1d_to_string(arr)
Converts 1D numpy array to string
- Parameters:
arr (Union[numpy.ndarray, list, tuple]) – 1D numpy array of numbers or strings to convert to string
- Returns:
String of combined values or the same value if not a list
- Return type:
Union[tuple, str, numbers.Number]
- hypso.utils.utils_file.recursive_print_nc(nc_file, path='', depth=0)
Navigate recursively the structure of a .nc file and print the structure. The function will call itself to navigate recursively.
- Parameters:
nc_file (netCDF4.Dataset) – Open .nc Dataset file
path (str) – Relative path to append to recursivness
depth (int) – Depth at which we are navigating
- Returns:
No return.
- Return type:
None
- hypso.utils.utils_file.find_closest_water_lat_lon_match(sat_obj, target_lat, target_lon, water_pixel_filter=False)
Find the closest Image lat and lon to the targat specificed that is also a water pixel.
- Parameters:
water_pixel_filter (bool) – If true, the closest pixel should also be classified as water pixel.
sat_obj – Hypso satellite object
target_lat (float) – Target latitude
target_lon (float) – Target longitude
- Returns:
Row, col and dist_to_target in km from the closes water pixel
- Return type:
Tuple[int, int, float]
- hypso.utils.utils_file.haversine(lon1, lat1, lon2, lat2)
Calculate the great circle distance between two points on the earth (specified in decimal degrees). Source: https://stackoverflow.com/a/4913653
- Parameters:
lon1 – Longitude of first point
lat1 – Latitude of first point
lon2 – Longitude of second point
lat2 – Latitude of second point
- Returns:
Distance in km between two points on earth