Added normalisation and unit test cases #118
Closed
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
Implemented multiple data transformation functions (normalize, scale, clipping, exponential, standardize, z_score_normalize) to enhance the framework's data processing capabilities. These functions are equipped with comprehensive unit tests to ensure correctness and handle edge cases.
Breakdown of Each Implemented Function
Normalize
Scales data to a specified range [min_value, max_value].
Includes edge case handling for zero range and empty data.
Scale
Scales data by a specified multiplier.
Useful for linear scaling transformations.
Clipping
Clips data to fall within a specified range [min_value, max_value].
Prevents extreme outliers in datasets.
Exponential Transformation
Applies an exponential transformation to data with a specified base.
Handles exponential growth scenarios effectively.
Standardize
Standardizes data to have a mean of 0 and a standard deviation of 1.
Includes custom mean and standard deviation parameters.
Z-Score Normalize
Computes z-scores for data points for standardization.
Handles mixed positive and negative datasets effectively.
Related Issue
Motivation and Context
The newly added functions provide robust data normalization and transformation capabilities, which are critical for preparing data for machine learning, statistical analysis, and other computational tasks. These functions solve the problem of inconsistent data scaling and ensure uniform preprocessing pipelines.
How Has This Been Tested?
Unit Tests: Added unit tests for each function:
Verified outputs for standard, edge, and invalid inputs.
Tests include large numbers, small numbers, mixed data types, and edge cases like empty datasets or identical values.
Environment: Testing performed on:
Python 3.12
OS: Windows 11
Commands:
Ran python -m unittest discover tests to ensure all test cases passed successfully.
Validated compatibility with existing project components.
Screenshots (if appropriate):
Types of changes
Checklist: