hypso.utils.utils_file module

hypso.utils.utils_file.HSI2RGB(wY, HSI, d=65, threshold=0.002)

Calculate a colore accurate RGB image with HSI values. Original function used for the paper “Creating RGB Images from Hyperspectral Images Using a Color Matching Function” by M. Magnusson, J. Sigurdsson, S. E. Armansson, M. O. Ulfarsson, H. Deborah and J. R. Sveinsson,

If you use this method, please cite the following paper: @INPROCEEDINGS{hsi2rgb, author={M. {Magnusson} and J. {Sigurdsson} and S. E. {Armansson} and M. O. {Ulfarsson} and H. {Deborah} and J. R. {Sveinsson}}, booktitle={IEEE International Geoscience and Remote Sensing Symposium}, title={Creating {RGB} Images from Hyperspectral Images using a Color Matching Function}, year={2020}, volume={}, number={}, pages={}}

Paper is available at https://www.researchgate.net/profile/Jakob_Sigurdsson

Parameters:

  • wY – Wavelengths in nm

  • HSI – Hyperspectral Image as a (#pixels x #bands) matrix,

  • d (Literal[50, 55, 65, 75]) – Determines the illuminant used, if in doubt use d65. Values of 50, 55, 65, 75

  • threshold – True if RGB thesholding should be done to increase contrast

Returns:

RGB Image array

Return type:

ndarray

class hypso.utils.utils_file.MyProgressBar(text_prefix)

Bases: object

Class used to display a progress bar on the screen for bulk downloading.

hypso.utils.utils_file.compare_netcdf_files(file1, file2)

Compare two .nc files and returns a pandas DataFrame where each attribute, variable and values are compared side by side.

Parameters:

  • file1 (str) – Absolute path to file 1

  • file2 (str) – Absolute path to file 2

Returns:

Dataframe with the compared characteristics

Return type:

DataFrame

hypso.utils.utils_file.find_all_files(path, str_in_file, suffix=None, type='partial')

Find recursively all files in a specified directory.

Parameters:

  • path (Path) – Directory path on which to look for files.

  • str_in_file (str) – Check if this string is in the file name

  • suffix (Optional[str]) – Check if file extension matches the suffix provided

  • type (Literal['partial', 'exact']) – Type of comparisson of “str_in_file”. If “partial”, then “str_in_file” will be checked to be in the filename, otherwise “exact” will check if the filename matches exactly the string provided.

Returns:

List of file paths

Return type:

list

hypso.utils.utils_file.find_closest_water_lat_lon_match(sat_obj, target_lat, target_lon, water_pixel_filter=False)

Find the closest Image lat and lon to the targat specificed that is also a water pixel.

Parameters:

  • water_pixel_filter (bool) – If true, the closest pixel should also be classified as water pixel.

  • sat_obj – Hypso satellite object

  • target_lat (float) – Target latitude

  • target_lon (float) – Target longitude

Returns:

Row, col and dist_to_target in km from the closes water pixel

Return type:

Tuple[int, int, float]

hypso.utils.utils_file.find_dir(path, str_in_dir, type='partial')

Find recursively a directory (or first occurance) in a specified directory.

Parameters:

  • path (Path) – Directory path on which to look for files.

  • str_in_dir (str) – Check if this string is in the directory name

  • type (Literal['partial', 'exact']) – Type of comparisson of “str_in_file”. If “partial”, then “str_in_file” will be checked to be in the filename, otherwise “exact” will check if the filename matches exactly the string provided.

Returns:

Path of the file or None if no file found

Return type:

Optional[Path]

hypso.utils.utils_file.find_file(path, str_in_file, suffix=None, type='partial')

Find recursively a single file (or first occurance) in a specified directory.

Parameters:

  • path (Path) – Directory path on which to look for files.

  • str_in_file (str) – Check if this string is in the file name

  • suffix (Optional[str]) – Check if file extension matches the suffix provided

  • type (Literal['partial', 'exact']) – Type of comparisson of “str_in_file”. If “partial”, then “str_in_file” will be checked to be in the filename, otherwise “exact” will check if the filename matches exactly the string provided.

Returns:

Path of the file or None if no file found

Return type:

Optional[Path]

hypso.utils.utils_file.flatten_dict(nested_dict)

Flatten a nested dictionary into a single level dictionary

Parameters:

nested_dict (dict) – Nested dictionary to flatten

Returns:

Single level dictionary with keys corresponding to the previous nested levels

Return type:

dict

hypso.utils.utils_file.haversine(lon1, lat1, lon2, lat2)

Calculate the great circle distance between two points on the earth (specified in decimal degrees). Source: https://stackoverflow.com/a/4913653

Parameters:

  • lon1 – Longitude of first point

  • lat1 – Latitude of first point

  • lon2 – Longitude of second point

  • lat2 – Latitude of second point

Returns:

Distance in km between two points on earth

hypso.utils.utils_file.is_integer_num(n)

Check if a number is an integer

Parameters:

n – Number to check

Returns:

Boolean value indicating whether or not the number is an integer

Return type:

bool

hypso.utils.utils_file.list_array_1d_to_string(arr)

Converts 1D numpy array to string

Parameters:

arr (Union[ndarray, list, tuple]) – 1D numpy array of numbers or strings to convert to string

Returns:

String of combined values or the same value if not a list

Return type:

Union[tuple, str, Number]

hypso.utils.utils_file.navigate_recursive_nc(nc_file, path='', depth=0)

Navigate recursively the structure of a .nc file and return the dictionary structure. The function will call itself to navigate recursively.

Parameters:

  • nc_file (Dataset) – Open .nc Dataset file

  • path (str) – Relative path to append to recursivness

  • depth (int) – Depth at which we are navigating

Returns:

Dictionary of dictionaries containing the structure of the .nc file

Return type:

dict

hypso.utils.utils_file.nested_dict_to_df(values_dict)

Convert nested dictionary to DataFrame.

Parameters:

values_dict (dict) – Nested dictionary

Returns:

Dataframe where the nested level are displayed as a column.

Return type:

DataFrame

hypso.utils.utils_file.print_nc(nc_file_path)

Print the contents of a .nc file :param nc_file_path: Absolute path to a .nc file

Returns:

No return

Parameters:

nc_file_path (str) –

Return type:

None

hypso.utils.utils_file.recursive_print_nc(nc_file, path='', depth=0)

Navigate recursively the structure of a .nc file and print the structure. The function will call itself to navigate recursively.

Parameters:

  • nc_file (Dataset) – Open .nc Dataset file

  • path (str) – Relative path to append to recursivness

  • depth (int) – Depth at which we are navigating

Returns:

No return.

Return type:

None