The challenge
A service-oriented company was preparing for a migration of their data. And, as a previous and critical step, they needed to clean and validate inventory data that came from different sources.
They brought us in to build an ETL process. This began as a technical task, but quickly evolved into a broader collaboration around data architecture and strategic recommendations.
What we did
The client was dealing with data that came from several sources, with different naming conventions and formats. Our job was to unify and prepare this data for future use.
Our solution: we built an ETL, which follows this process:
- Multi-source ingestion: We ingest data from files in Excel and CSV, coming from different sources to Microsoft Fabric.
- Rule-based cleaning: We developed logic and coded rules for this cleaning stage.
- Business validation: The results, with the clean data, are stored on Excel tables and visualized on PowerBI dashboards. The user receives a table with all the data for validation, according to business rules, and we constantly iterate through the process.
What we achieved
Our team succeeded in building a useful ETL for preparing, cleaning, and validating the data prior to the next step: migration.
We implemented the process and, although it was a challenge to work with Microsoft Fabric (since it is a relatively recent product and some of its functions were still in the early steps), it was a successful project: we established the Data Cleansing flow with great results and positive feedback from the client.
This project is a strong example of how data preparation and cleaning ensure long-term success in a strong data architecture.
“The migration of our system would not have been possible without the assistance of the zenital team. Thanks to the ETL they built to clean and validate our data, and to their assistance in applying business rules, we now have better data quality and a clear and structured process, optimizing our work”.
The challenge
A service-oriented company was preparing for a migration of their data. And, as a previous and critical step, they needed to clean and validate inventory data that came from different sources.
They brought us in to build an ETL process. This began as a technical task, but quickly evolved into a broader collaboration around data architecture and strategic recommendations.
What we did
The client was dealing with data that came from several sources, with different naming conventions and formats. Our job was to unify and prepare this data for future use.
Our solution: we built an ETL, which follows this process:
- Multi-source ingestion: We ingest data from files in Excel and CSV, coming from different sources to Microsoft Fabric.
- Rule-based cleaning: We developed logic and coded rules for this cleaning stage.
- Business validation: The results, with the clean data, are stored on Excel tables and visualized on PowerBI dashboards. The user receives a table with all the data for validation, according to business rules, and we constantly iterate through the process.
How to change an entire data strategy
I imagine 8wires as that sincere and honest partner that takes you out of all that noise and helps you focus on what is important, no matter how unsexy it may be, to achieve a great long-term goal.
The one who accompanies you through the hard times and helps you through the tough decisions, knowing that there is no easy road. The one who gives you the push or the tools so that you climb and be yourself the one who reaches the summits you propose in a healthy, sustainable and energetic way. And above all, the one who steps aside when he knows that he is not helping you or that he will not be able to give you what you need.
I don't know if it is helpful, but somehow I saw on the web a visual explanation of the problem in the data/technology world (I don't know if with this metaphor) before showing how it is to work with us and finally, another visual explanation of the result.
Like a bridge over troubled waters
I imagine 8wires as that sincere and honest partner that takes you out of all that noise and helps you focus on what is important, no matter how unsexy it may be, to achieve a great long-term goal.
The one who accompanies you through the hard times and helps you through the tough decisions, knowing that there is no easy road. The one who gives you the push or the tools so that you climb and be yourself the one who reaches the summits you propose in a healthy, sustainable and energetic way.
The challenge
A service-oriented company was preparing for a migration of their data. And, as a previous and critical step, they needed to clean and validate inventory data that came from different sources.
They brought us in to build an ETL process. This began as a technical task, but quickly evolved into a broader collaboration around data architecture and strategic recommendations.
What we did
The client was dealing with data that came from several sources, with different naming conventions and formats. Our job was to unify and prepare this data for future use.
Our solution: we built an ETL, which follows this process:
- Multi-source ingestion: We ingest data from files in Excel and CSV, coming from different sources to Microsoft Fabric.
- Rule-based cleaning: We developed logic and coded rules for this cleaning stage.
- Business validation: The results, with the clean data, are stored on Excel tables and visualized on PowerBI dashboards. The user receives a table with all the data for validation, according to business rules, and we constantly iterate through the process.
How to change an entire data strategy
I imagine 8wires as that sincere and honest partner that takes you out of all that noise and helps you focus on what is important, no matter how unsexy it may be, to achieve a great long-term goal.
The one who accompanies you through the hard times and helps you through the tough decisions, knowing that there is no easy road. The one who gives you the push or the tools so that you climb and be yourself the one who reaches the summits you propose in a healthy, sustainable and energetic way. And above all, the one who steps aside when he knows that he is not helping you or that he will not be able to give you what you need.
I don't know if it is helpful, but somehow I saw on the web a visual explanation of the problem in the data/technology world (I don't know if with this metaphor) before showing how it is to work with us and finally, another visual explanation of the result.
Like a bridge over troubled waters
I imagine 8wires as that sincere and honest partner that takes you out of all that noise and helps you focus on what is important, no matter how unsexy it may be, to achieve a great long-term goal.
The one who accompanies you through the hard times and helps you through the tough decisions, knowing that there is no easy road. The one who gives you the push or the tools so that you climb and be yourself the one who reaches the summits you propose in a healthy, sustainable and energetic way.
.png){kind=small}