preprocess

_preprocess

This file is a base class for preprocessing module.

class scalr.data.preprocess._preprocess.PreprocessorBase(**kwargs)

Bases: object

Base class for preprocessor

fit(data: AnnData | AnnCollection, sample_chunksize: int) → None

A function to calculate attributes for transformation.

It is required only when you need to see the entire train data and calculate attributes, as required in StdScaler, etc. This method should not return anything, it should be used to store attributes that will be used by the transform method.

Parameters:

• data (Union[AnnData, AnnCollection]) – train_data in backed mode.

• sample_chunksize (int) – several samples of data that can at most be loaded in memory.

process_data(full_data: AnnData | AnnCollection, sample_chunksize: int, dirpath: str, num_workers: int = 1)

A function to process the entire data chunkwise and write the processed data to disk.

Parameters:

• full_data (Union[AnnData, AnnCollection]) – Full data for transformation.

• sample_chunksize (int) – Number of samples in one chunk.

• dirpath (str) – Path to write the data to.

• num_workers (int) – number of jobs to run in parallel for data writing.

transform(data: ndarray) → ndarray

A required function to transform a numpy array.

Parameters:

data (np.ndarray) – Input raw data.

Returns:

Processed data.

Return type:

np.ndarray

scalr.data.preprocess._preprocess.build_preprocessor(preprocessing_config: dict) → tuple[PreprocessorBase, dict]

Builder object to get a processor, updated preprocessing_config.

sample_norm

This file performs Sample-wise normalization on the data.

class scalr.data.preprocess.sample_norm.SampleNorm(scaling_factor: float = 1.0)

Bases: PreprocessorBase

Class for Samplewise Normalization

classmethod get_default_params() → dict

Class method to get default params for preprocess_config.

transform(data: ndarray) → ndarray

A function to transform provided input data.

Parameters:

data (np.ndarray) – Input raw data.

Returns:

Processed data.

Return type:

np.ndarray

standard_scale

This file performs standard scaler normalization on the data.

class scalr.data.preprocess.standard_scale.StandardScaler(with_mean: bool = True, with_std: bool = True)

Bases: PreprocessorBase

Class for Standard Normalization

calculate_mean(data: AnnData | AnnCollection, sample_chunksize: int) → None

Function to calculate mean for each feature in the train data

Parameters:

• data – Data to calculate the mean of.

• sample_chunksize – Chunks of data that can be loaded into memory at once.

Returns:

Nothing, stores mean per feature of the train data.

calculate_std(data: AnnData | AnnCollection, sample_chunksize: int) → None

A function to calculate standard deviation for each feature in the train data.

Parameters:

• data – Data to calculate the standard deviation of

• sample_chunksize – Chunks of data that can be loaded into memory at once.

Returns:

Nothing, stores standard deviation per feature of the train data.

fit(data: AnnData | AnnCollection, sample_chunksize: int) → None

This function calculate parameters for standard scaler object from the train data.

Parameters:

• data – Data to calculate the required parameters of.

• sample_chunksize – Chunks of data that can be loaded into memory at once.

classmethod get_default_params() → dict

Class method to get default params for preprocess_config.

transform(data: ndarray) → ndarray

A function to transform provided input data.

Parameters:

data (np.ndarray) – raw data

Returns:

processed data

Return type:

np.ndarray

test_sample_norm

This is a test file for Sample-wise normalization.

scalr.data.preprocess.test_sample_norm.test_transform()

This function tests the transform function of Sample-wise normalization.

There is no fit() involved in Sample-wise normalization.

test_standard_scale

This is a test file for standard-scaler normalization.

scalr.data.preprocess.test_standard_scale.test_fit()

This function tests fit() function of sample-norm normalization.

fit() function is enough for testing, as we can compare mean and std with sklean standard-scaler object params.