Page History
Yellowfin comes with a set of in-built transformation steps, but additional steps can also be incorporated that will apply data science models to data. This is done via special plug-ins available on the Marketplace that support numerous data science outputs such as PMML, PFA, R and H2O. Further transactions can also be performed on data by applying various functions through
The Data Transformation module offers different ways of performing transformations on your data. These include transformation steps that can be dragged into a flow, performing inline field transformations, and even data conversion. Besides the transformation steps that come included with this module, you can also download and install plug-ins to include more steps in your environment.
Here is a breakdown of the different ways of performing transformations:
- Built-in transformations: This refers to transformations in the transformation step panel that come included in the data transformation module. These do not require any type of set up.
- Downloadable transformation step: You can also download and install special plug-ins from the Marketplace that become available in the transformation step panel. (See the Transform Steps section in the Marketplace for all available options.)
- Inline field transformations: This refers to functions that users can perform directly on the data (in the data preview panel or the
...
- step configuration panel).
Following is a list of all the transformations that you can perform on your data. Click on a transformation name to learn how to use it in your flow.
Transformation | Related ComponentType | Description | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| InBuilt-built in transformation step | This step transforms your data into a summary form, by applying functions like count, count distinct, sum, average, etc. | ||||||||||
| InBuilt-built in transformation step | This step creates a calculated field based on other fields in the step. | ||||||||||
| InBuilt-built in transformation step | This step is for filtering data in a step. | ||||||||||
| InBuilt-built in transformation step | This step merges two sets of data based on the configured Join Fields. | ||||||||||
| InBuilt-built in transformation step | This step duplicates an input dataset to create identical output datasets. | ||||||||||
| Union | Built-in transformation step | This step combines the data from two steps together. | |||||||||
| Date Component | Built-in transformation step | This step extracts specific date elements from date fields. | |||||||||
| PitneyBowes Forward Geocoding | Downloadable transformation step | This step uses the PitneyBowes forward geocoding API to convert address values into location coordinates. | |||||||||
| OpenCage Forward Geocoding | Downloadable transformation step | Uses the OpenCage forward geocoding API to convert address values into location coordinates. | |||||||||
| PMML Model Prediction | Downloadable transformation step | This step integrates a model saved as a PMML model file into the platform and applies it on the data. | |||||||||
| PFA Model Prediction | Downloadable transformation step | This step integrates a PFA model and applies it on the data. | |||||||||
| H2O Model Prediction | Downloadable transformation step | This step integrates a model created in H2O model .ai and applies it on the data. | |||||||||
| R Model Prediction | Downloadable transformation step | This step integrates a an R model and applies it on the data. | |||||||||
| Datatype ConversionData Type Conversions | Inline Field transformation | Set of functions used to convert the data type of a field into another type. For example, text can be converted into numeric values, SQL date, or SQL timestamp. | |||||||||
| GeoPoint Conversion | Data type conversion | Generate GeoPoint values based on geographical data. | |||||||||
| Duplicate Field |
| Create a copy of the data field. | Duplicate Field | Field transformation | |||||||
| Number Precision | Inline Field transformation | Round off a numeric value. | |||||||||
| Sub-string | Inline Field transformation | Extract part of a string. | |||||||||
| Switch Case | Inline Field transformation | Convert text to all uppercase, all lowercase, or proper case. | |||||||||
| White Space | Inline Field transformation | Remove white space from data. | |||||||||
| Find and Replace | Inline Field transformation | Search for specific data values and replace them with other values. | |||||||||
| Grouped Data | Inline Field transformation | Group a field's values. | |||||||||
| Nulls to Zero | Inline Field transformation | Replaces every null value in a numeric field with zero. |