sasdata.data_util package¶
Submodules¶
sasdata.data_util.err1d module¶
Error propogation algorithms for simple arithmetic
Warning: like the underlying numpy library, the inplace operations may return values of the wrong type if some of the arguments are integers, so be sure to create them with floating point inputs.
- sasdata.data_util.err1d.add(X, varX, Y, varY)¶
Addition with error propagation
- sasdata.data_util.err1d.add_inplace(X, varX, Y, varY)¶
In-place addition with error propagation
- sasdata.data_util.err1d.div(X, varX, Y, varY)¶
Division with error propagation
- sasdata.data_util.err1d.div_inplace(X, varX, Y, varY)¶
In-place division with error propagation
- sasdata.data_util.err1d.exp(X, varX)¶
Exponentiation with error propagation
- sasdata.data_util.err1d.log(X, varX)¶
Logarithm with error propagation
- sasdata.data_util.err1d.mul(X, varX, Y, varY)¶
Multiplication with error propagation
- sasdata.data_util.err1d.mul_inplace(X, varX, Y, varY)¶
In-place multiplication with error propagation
- sasdata.data_util.err1d.pow(X, varX, n)¶
X**n with error propagation
- sasdata.data_util.err1d.pow_inplace(X, varX, n)¶
In-place X**n with error propagation
- sasdata.data_util.err1d.sub(X, varX, Y, varY)¶
Subtraction with error propagation
- sasdata.data_util.err1d.sub_inplace(X, varX, Y, varY)¶
In-place subtraction with error propagation
sasdata.data_util.formatnum module¶
Format values and uncertainties nicely for printing.
format_uncertainty_pm()
produces the expanded format v +/- err.
format_uncertainty_compact()
produces the compact format v(##),
where the number in parenthesis is the uncertainty in the last two digits of v.
format_uncertainty()
uses the compact format by default, but this
can be changed to use the expanded +/- format by setting
format_uncertainty.compact to False.
The formatted string uses only the number of digits warranted by the uncertainty in the measurement.
If the uncertainty is 0 or not otherwise provided, the simple %g floating point format option is used.
Infinite and indefinite numbers are represented as inf and NaN.
Example:
>>> v,dv = 757.2356,0.01032
>>> print format_uncertainty_pm(v,dv)
757.236 +/- 0.010
>>> print format_uncertainty_compact(v,dv)
757.236(10)
>>> print format_uncertainty(v,dv)
757.236(10)
>>> format_uncertainty.compact = False
>>> print format_uncertainty(v,dv)
757.236 +/- 0.010
UncertaintyFormatter() returns a private formatter with its own formatter.compact flag.
- sasdata.data_util.formatnum.format_uncertainty_compact(value, uncertainty)¶
Given value v and uncertainty dv, return the compact representation v(##), where ## are the first two digits of the uncertainty.
- sasdata.data_util.formatnum.format_uncertainty_pm(value, uncertainty)¶
Given value v and uncertainty dv, return a string v +/- dv.
sasdata.data_util.loader_exceptions module¶
Exceptions specific to loading data.
- exception sasdata.data_util.loader_exceptions.DataReaderException(e: str | None = None)¶
Bases:
Exception
Exception for files that were able to mostly load, but had minor issues along the way. Any exceptions of this type should be put into the datainfo.errors
- exception sasdata.data_util.loader_exceptions.DefaultReaderException(e: str | None = None)¶
Bases:
Exception
Exception for files with no associated reader. This should be thrown by default readers only to tell Loader to try the next reader.
- exception sasdata.data_util.loader_exceptions.FileContentsException(e: str | None = None)¶
Bases:
Exception
Exception for files with an associated reader, but with no loadable data. This is useful for catching loader or file format issues.
- exception sasdata.data_util.loader_exceptions.NoKnownLoaderException(e: str | None = None)¶
Bases:
Exception
Exception for files with no associated reader based on the file extension of the loaded file. This exception should only be thrown by loader.py.
sasdata.data_util.manipulations module¶
Data manipulations for 2D data sets. Using the meta data information, various types of averaging are performed in Q-space
To test this module use:
`
cd test
PYTHONPATH=../src/ python2 -m sasdataloader.test.utest_averaging DataInfoTests.test_sectorphi_quarter
`
- class sasdata.data_util.manipulations.Binning(min_value, max_value, n_bins, base=None)¶
Bases:
object
This class just creates a binning object either linear or log
- get_bin_index(value)¶
- Parameters:
value – the value in the binning interval whose bin index should be returned. Must be between min_value and max_value.
The general formula logarithm binning is: bin = floor(N * (log(x) - log(min)) / (log(max) - log(min)))
- class sasdata.data_util.manipulations.Boxavg(x_min=0.0, x_max=0.0, y_min=0.0, y_max=0.0)¶
Bases:
Boxsum
Perform the average of counts in a 2D region of interest.
- class sasdata.data_util.manipulations.Boxcut(x_min=0.0, x_max=0.0, y_min=0.0, y_max=0.0)¶
Bases:
object
Find a rectangular 2D region of interest.
- class sasdata.data_util.manipulations.Boxsum(x_min=0.0, x_max=0.0, y_min=0.0, y_max=0.0)¶
Bases:
object
Perform the sum of counts in a 2D region of interest.
- class sasdata.data_util.manipulations.CircularAverage(r_min=0.0, r_max=0.0, bin_width=0.0005)¶
Bases:
object
Perform circular averaging on 2D data
The data returned is the distribution of counts as a function of Q
- class sasdata.data_util.manipulations.Ring(r_min=0, r_max=0, center_x=0, center_y=0, nbins=36)¶
Bases:
object
Defines a ring on a 2D data set. The ring is defined by r_min, r_max, and the position of the center of the ring.
The data returned is the distribution of counts around the ring as a function of phi.
Phi_min and phi_max should be defined between 0 and 2*pi in anti-clockwise starting from the x- axis on the left-hand side
- class sasdata.data_util.manipulations.Ringcut(r_min=0, r_max=0, center_x=0, center_y=0)¶
Bases:
object
Defines a ring on a 2D data set. The ring is defined by r_min, r_max, and the position of the center of the ring.
The data returned is the region inside the ring
Phi_min and phi_max should be defined between 0 and 2*pi in anti-clockwise starting from the x- axis on the left-hand side
- class sasdata.data_util.manipulations.SectorPhi(r_min, r_max, phi_min=0, phi_max=6.283185307179586, nbins=20, base=None)¶
Bases:
_Sector
Sector average as a function of phi. I(phi) is return and the data is averaged over Q.
A sector is defined by r_min, r_max, phi_min, phi_max. The number of bin in phi also has to be defined.
- class sasdata.data_util.manipulations.SectorQ(r_min, r_max, phi_min=0, phi_max=6.283185307179586, nbins=20, base=None)¶
Bases:
_Sector
Sector average as a function of Q for both wings. setting the _Sector.fold attribute determines whether or not the two sectors are averaged together (folded over) or separate. In the case of separate (not folded), the qs for the “minor wing” are arbitrarily set to a negative value. I(Q) is returned and the data is averaged over phi.
A sector is defined by r_min, r_max, phi_min, phi_max. where r_min, r_max, phi_min, phi_max >0. The number of bin in Q also has to be defined.
- class sasdata.data_util.manipulations.Sectorcut(phi_min=0, phi_max=3.141592653589793)¶
Bases:
object
Defines a sector (major + minor) region on a 2D data set. The sector is defined by phi_min, phi_max, where phi_min and phi_max are defined by the right and left lines wrt central line.
Phi_min and phi_max are given in units of radian and (phi_max-phi_min) should not be larger than pi
- class sasdata.data_util.manipulations.SlabX(x_min=0.0, x_max=0.0, y_min=0.0, y_max=0.0, bin_width=0.001, fold=False)¶
Bases:
_Slab
Compute average I(Qx) for a region of interest
- class sasdata.data_util.manipulations.SlabY(x_min=0.0, x_max=0.0, y_min=0.0, y_max=0.0, bin_width=0.001, fold=False)¶
Bases:
_Slab
Compute average I(Qy) for a region of interest
- sasdata.data_util.manipulations.flip_phi(phi: float) float ¶
Force phi to be within the 0 <= to <= 2pi range by adding or subtracting 2pi as necessary
- Returns:
phi in >=0 and <=2Pi
- sasdata.data_util.manipulations.get_dq_data(data2d: Data2D) array ¶
Get the dq for resolution averaging The pinholes and det. pix contribution present in both direction of the 2D which must be subtracted when converting to 1D: dq_overlap should calculated ideally at q = 0. Note This method works on only pinhole geometry. Extrapolate dqx(r) and dqy(phi) at q = 0, and take an average.
- sasdata.data_util.manipulations.get_intercept(q: float, q_0: float, q_1: float) float | None ¶
Returns the fraction of the side at which the q-value intercept the pixel, None otherwise. The values returned is the fraction ON THE SIDE OF THE LOWEST Q.
A B +-----------+--------+ <--- pixel size 0 1 Q_0 -------- Q ----- Q_1 <--- equivalent Q range if Q_1 > Q_0, A is returned if Q_1 < Q_0, B is returned if Q is outside the range of [Q_0, Q_1], None is returned
- sasdata.data_util.manipulations.get_pixel_fraction(q_max: float, q_00: float, q_01: float, q_10: float, q_11: float) float ¶
Returns the fraction of the pixel defined by the four corners (q_00, q_01, q_10, q_11) that has q < q_max.:
q_01 q_11 y=1 +--------------+ | | | | | | y=0 +--------------+ q_00 q_10 x=0 x=1
- sasdata.data_util.manipulations.get_pixel_fraction_square(x: float, x_min: float, x_max: float) float ¶
Return the fraction of the length from xmin to x.:
A B +-----------+---------+ xmin x xmax
- Parameters:
x – x-value
x_min – minimum x for the length considered
x_max – minimum x for the length considered
- Returns:
(x-xmin)/(xmax-xmin) when xmin < x < xmax
- sasdata.data_util.manipulations.get_q_compo(dx: float, dy: float, detector_distance: float, wavelength: float, compo: str | None = None) float ¶
This reduces tiny error at very large q. Implementation of this func is not started yet.<–ToDo
- sasdata.data_util.manipulations.position_and_wavelength_to_q(dx: float, dy: float, detector_distance: float, wavelength: float) float ¶
- Parameters:
dx – x-distance from beam center [mm]
dy – y-distance from beam center [mm]
detector_distance – sample to detector distance [mm]
wavelength – neutron wavelength [nm]
- Returns:
q-value at the given position
sasdata.data_util.nxsunit module¶
Define unit conversion support for NeXus style units.
The unit format is somewhat complicated. There are variant spellings and incorrect capitalization to worry about, as well as forms such as “mili*metre” and “1e-7 seconds”.
This is a minimal implementation of units including only what I happen to need now. It does not support the complete dimensional analysis provided by the package udunits on which NeXus is based, or even the units used in the NeXus definition files.
Unlike other units packages, this package does not carry the units along with the value but merely provides a conversion function for transforming values.
Usage example:
import nxsunit
u = nxsunit.Converter('mili*metre') # Units stored in mm
v = u(3000,'m') # Convert the value 3000 mm into meters
NeXus example:
# Load sample orientation in radians regardless of how it is stored.
# 1. Open the path
file.openpath('/entry1/sample/sample_orientation')
# 2. scan the attributes, retrieving 'units'
units = [for attr,value in file.attrs() if attr == 'units']
# 3. set up the converter (assumes that units actually exists)
u = nxsunit.Converter(units[0])
# 4. read the data and convert to the correct units
v = u(file.read(),'radians')
This is a standalone module, not relying on either DANSE or NeXus, and can be used for other unit conversion tasks.
Note: minutes are used for angle and seconds are used for time. We cannot tell what the correct interpretation is without knowing something about the fields themselves. If this becomes an issue, we will need to allow the application to set the dimension for the unit rather than inferring the dimension from an example unit.
- class sasdata.data_util.nxsunit.Converter(units: str | None = None, dimension: List[str] | None = None)¶
Bases:
object
Unit converter for NeXus style units.
The converter is initialized with the units of the source value. Various source values can then be converted to target values based on target value name.
- dimension: List[str] = None¶
Type of the source units (distance, time, frequency, …)
- get_compatible_units() List[str] ¶
Return a list of compatible units for the current Convertor object
- scale(units: str = '', value: T = None) List[float] | T ¶
Scale the given value using the units string supplied
- scalebase: float = None¶
Scale base for the source units
- scalemap: List[Dict[str, float | Tuple[float, float]]] = None¶
Scale converter, mapping unit name to scale factor or (scale, offset) for temperature units.
- scaleoffset: float = None¶
- property units: str¶
- sasdata.data_util.nxsunit.standardize_units(unit: str | None) List[str] ¶
Convert supplied units to a standard format for maintainability :param unit: Raw unit as supplied :return: Unit with known, reduced values
sasdata.data_util.registry module¶
File extension registry.
This provides routines for opening files based on extension, and registers the built-in file extensions.
- class sasdata.data_util.registry.CustomFileOpen(filename, mode='rb')¶
Bases:
object
Custom context manager to fetch file contents depending on where the file is located.
- class sasdata.data_util.registry.ExtensionRegistry¶
Bases:
object
Associate a file loader with an extension.
Note that there may be multiple loaders for the same extension.
Example:
registry = ExtensionRegistry() # Add an association by setting an element registry['.zip'] = unzip # Multiple extensions for one loader registry['.tgz'] = untar registry['.tar.gz'] = untar # Generic extensions to use after trying more specific extensions; # these will be checked after the more specific extensions fail. registry['.gz'] = gunzip # Multiple loaders for one extension registry['.cx'] = cx1 registry['.cx'] = cx2 registry['.cx'] = cx3 # Show registered extensions print registry.extensions() # Can also register a format name for explicit control from caller registry['cx3'] = cx3 print registry.formats() # Retrieve loaders for a file name registry.lookup('hello.cx') -> [cx3,cx2,cx1] # Run loader on a filename registry.load('hello.cx') -> try: return cx3('hello.cx') except: try: return cx2('hello.cx') except: return cx1('hello.cx') # Load in a specific format ignoring extension registry.load('hello.cx',format='cx3') -> return cx3('hello.cx')
- extensions() List[str] ¶
Return a sorted list of registered extensions.
- formats() List[str] ¶
Return a sorted list of the registered formats.
- load(path: str, ext: str | None = None) List[Data1D | Data2D] ¶
Call the loader for a single file.
Exceptions are stored in Data1D instances, with the errors in Data1D.errors
- lookup(path: str) List[callable] ¶
Return the loader associated with the file type of path.
- Parameters:
path – Data file path
- Returns:
List of available readers for the file extension (maybe empty)
- sasdata.data_util.registry.create_empty_data_with_errors(path: str | Path, errors: List[Exception])¶
Create a Data1D instance that only holds errors and a filepath. This allows all file paths to return a common data type, regardless if the data loading was successful or a failure.
sasdata.data_util.uncertainty module¶
Uncertainty propagation class for arithmetic, log and exp.
Based on scalars or numpy vectors, this class allows you to store and manipulate values+uncertainties, with propagation of gaussian error for addition, subtraction, multiplication, division, power, exp and log.
Storage properties are determined by the numbers used to set the value and uncertainty. Be sure to use floating point uncertainty vectors for inplace operations since numpy does not do automatic type conversion. Normal operations can use mixed integer and floating point. In place operations such as a *= b create at most one extra copy for each operation. By contrast, c = a*b uses four intermediate vectors, so shouldn’t be used for huge arrays.
sasdata.data_util.util module¶
- sasdata.data_util.util.unique_preserve_order(seq: List[Any]) List[Any] ¶
Remove duplicates from list preserving order Fastest according to benchmarks at https://www.peterbe.com/plog/uniqifiers-benchmark