Platform

New Features

1. GitHub Integration: Provided an option to integrate GIT Hub under the New Version Control admin option along with Git Lab.
2. Data Store Settings: Provided the ClickHouse Data Store support for the Story module.
3. Data Center: Provided an option to create Data Store from the Sandbox File on the Sandbox List page.
4. SSL implementation for MySQL, Postgres, and ClickHouse.
5. Admin Module:
1. Execution Settings
2. Base URL Settings
3. Upload Certificate Settings
6. Breadcrumbs UI provided for the Admin and Data Center modules.
7. Reupload Sandbox file option has been provided.

Enhancements

1. Data Preparation updates:
1. The skip rows feature has been provided to skip the set number of rows for a Sandbox file.
2. Provided the following Advanced Transforms:
1. SQL
2. Expression Editor
3. Find Anomaly
4. Cluster and Edit
2. UI/UX changes and enhancements for the Platform Homepage.

Please Note: The BDB Platform is compliant with OWASP security standards.

Data Pipeline

New Features

1. ETL Jobs: Introduced the Spark ETL Job workflow from data ingestion to data transfer facilitating even data transformations and data cleansing inside it to avail faster data processing.
2. New Components
1. Sandbox Reader: Provided Sandbox Reader inside the Readers category to read data from the Sandbox files.
2. Data Preparation: Introduced a new component to perform data cleansing inside the Data Pipeline module.
3. Schema Validator: Introduced Schema Validator to validate schema.
3. Scripting components: Code editor UI has been provided.
4. Enrichment Component: Provided a new feature to Query and fetch the master table with an option to refresh the master table data on a given interval [refresh rate].

Enhancements

1. Kafka Producers has been enhanced to produce messages to external brokers.
2. WebSocket Producer: Implemented Path and secrets for security and data segregation in the WebSocket Producer component.
3. Mongo Reader Lite (Docker): Performance and resource utilization enhancements are done.
4. Kafka consumer Component: Host Aliases are available for both with and without SSL connections.
5. Spark SQL: Provided streaming Aggregation enhancements.
6. SSL implementation on Postgres and Mongo drivers in the respective components and DB sync
7. Enhanced Pipeline Monitoring UI.
8. Enhanced the DB Sync component in the following aspects:
1. Performance enhancements
2. UI Enhancements
3. SSL implementation
4. Snowflake Driver
9. RDBMS Reader/Writer: Snowflake driver has been added.

Data Science Lab

New Features

1. Model Explainability: Enabled the Model Explainability option for the Classification and Regression models to explore the models.
2. Model Migration:
1. Data Science Lab models can be migrated across spaces and servers (This includes the migration of Sklearn, Keras, and PyTorch models).
2. All dependent files will get shared into a new environment and work as it is.
3. Notebook Migration:
1. Data Science Lab notebooks can be migrated across spaces and servers.
2. Datasets, data preparations, utility files, and secrets used inside the notebook will also get migrated along with it.
3. All dependent files will get shared into a new environment and work as it is.
4. AutoML Performance Improvement: There is more than 30X improvement in the AutoML performance.
5. AutoML Model Inference in the Data Pipeline: The user can register all the top 3 AutoML models to the Data Pipeline and do the inferencing.
6. Import Model Functionality: External models: .pkl, .h5, .pt, .pth, .bin can be imported into the Data Science Lab and used for inferencing.
7. Implemented the PySpark environment in the Data Science Lab.

Enhancements

1. Variable Explorer: Provided a tab on the Notebook page to list the Variable Explorer information.
2. Datasets with Data Preparations can be used for creating AutoML experiments.
3. Upload Sandbox File: Provided an option to upload the sandbox files from the local directory to the Datasets tab.
4. Resource Utilization Graphs: Provided a resource utilization graph to indicate the latest usage of Memory & CPU on the Notebook page.
5. Preview Notebook: Provided an option to preview the Notebook without activating the Project of which the concerned Notebook is a part.
6. Implemented Drawers instead of pop-ups to display functionalities and related actions in the Data Science Lab module.
7. Copy Path option: The user can generate the Copy path for Sandbox files inside the Data Science Lab.

Designer

New Features

1. Dashboard size optimization & performance.
1. Updated all chart property files as a single file.
2. Metadata files are addressed as single files.
3. Split JSON: Separated the Components JSON from the Dashboard Content JSON.
2. Dashboard Template:: Provided an option to Save as Template in the More Actions dropdown of each dashboard on the Manage page. The created Template can be previewed or opened as a dashboard.
3. Alert User Notification: It sends an Alert to other users with a notification when one user saves the dashboard when multiple users are working on the same dashboard.
4. WebSocket
1. Added GUID, Ingestion id, and Ingestion secret in connector properties.
2. WebSocket triggers continuously.
5. New Charting components:
1. Input-Text component:
1. Provided option to select different types of texts (number, text, mail, etc.)
2. Provided the properties to control the styling.
2. Button Component:
1. Provided options for different types of default designs for buttons (primary, secondary, success, Danger, etc.).
2. Provided other design properties.
3. Decomposition chart: It automatically aggregates data and enables drilling down into your dimensions in the given order.
4. Tab Component: The Tab component contains the option to create multiple buttons. Each button has the option to associate visible and invisible groups (show/hide groups).
1. Tab with icon reference.
2. Provided Styling properties.

List of Enhancements

1. Copy to component:
1. Provided the option to duplicate the selected component.
2. Added option to duplicate the group.
2. Leaflet marker & Cluster type:
1. Provided option to add SVG icons.
3. Grid Components:
1. Data Grid
1. Added property to rearrange the columns.
2. Enables show data option when alert indicator as color fill.
2. Paging Grid: Added Fit column property.
3. Paging & Scorecard- Provided Font styling property for column.
4. Export Button:Added PDF format as the default selection. Export based on the dashboard dimension.
5. Chart Data Label:
1. Added property to Hide the data label value.
2. Added property to change the highest bar data label color.
6. Radio Button:
1. Implemented Enhance-Button type & Property control.
2. Added property option to control the circle size & color for radio type.
7. Group bar chart: Implemented the Text Wrap option for Category in the data.
8. List Filter: Added property to control Row Height & Action-Bar Height together with the color update.
9. Timeseries chart
1. Added a property to control the opacity of the series color when the chart type is a column.
2. Added property to control fill area color when chart type is a line.
10. Dashboard Scaling:
1. Added align property to set to the top when fit to width is enabled.
2. Added align property to set to left when fit to height is enabled.
11. Implemented Freeze Header as SDK method.
12. Date Picker:
1. Select date range as property.
2. Added property to control background color.
13. Label:
1. Added property to control hover color.
2. Added property to control border color & thickness.
14. Combo Filter: Provided a properties option to control background color.

Story

New Features

1. The complete UI of the Story module has been updated to Angular now.
2. File Upload:The Story module supports Excel files to create Data Store for creating a new Story.
1. The needs to be in the Sandbox and the data should be uploaded in chunks.
2. The profiling of data gets displayed to the end user for the file upload.
3. Implemented ClickHouse as a Data Store option to create a Story.
4. Data Profiling:
1. The user can get the data profile by hovering the mouse on the Data (measures or dimensions).
2. It displays the Total Unique count, total duplicate count, minimum variance, and standard deviation for Measures.
5. All Data Stores: New display has been provided for the All Data Store option. The location for this option has been moved to the right-side panel.
6. Formula Field: This option has been shifted to the right-side panel renamed as Create Calculated Field.
7. UI Updates:
1. Dimensions/Measures get added to the selected Dimension/Measure box by using double clicks on them.
2. The category value gets highlighted when drilled into it.
3. KPI Tile & Sparkline: Value and Label get top aligned.
4. Storyboard: Indicates the applied filters on the specific view. <
5. The enhanced design of the dialog boxes that appear across the Story module.
8. New Charting Components:
1. Funnel chart: It is a specialized chart type that demonstrates the flow of users through a business or sales process.
1. The chart gets plotted with 1 Dimension and 1 Measure.
2. The view is good when the selected dimension values are less than 10 unique values.
2. Inverted Funnel Chart:An Inverted Funnel chart is a specialized chart type that demonstrates the flow of users through a business or sales process.
1. The chart gets plotted with 1 Dimension and 1 Measure.
2. The view is good when the selected dimension values are less than 10 unique values.
3. Geo chart:Geo coordinates get supported through this charting component in the Story module.
1. The supported map types are:
1. Leaflet
2. SVG
4. Heat Map chart: This charting component helps to visualize complex data such as performance comparison of different companies, market response, stock market investments, etc.
9. Data Search updates: The Data Search supports Meta Data Stores and File Data Stores for searching across the data.

Enhancements

1. Gauge chart:
1. Provided Basic number formatting
2. Improved default look & feel of Gauge components like UI of Needle, Needle Pointer, Gradient Color, etc.
2. Decomposition chart: Bringing a new chart implemented by the Designer team into the Story module replacing the existing Dissolution chart.
3. Loading Data in Data Stores: Created Python Jobs to load the Data Store by reducing the resources and increasing the performance.