The Company
Cisco Systems, a Fortune 500 company founded in 1984, excels in networking, telecom, and software. With 84,900 employees worldwide, Cisco leads in network security and cloud services. Its operations span infrastructure platforms, applications, security, and more. Cisco’s market cap is around $57Bn, reflecting its formidable presence in the tech industry.
The Challenge
Cisco faced a complex challenge with its internal migration initiative, needing a data solution that met high standards for security and scalability. The company required an automated tool to avoid the time-consuming and resource-intensive manual process. Data security was a top priority, demanding no data movement during validation. Additionally, Cisco sought a scalable tool that could be leveraged organization-wide to ensure maximum output.
The Solution
- As a global powerhouse with operations spanning the globe, Cisco manages an immense volume of data.The initial challenge was devising a strategy to handle the project’s vast scope. The teams adopted a segmented strategy, breaking down the project into manageable phases for proper management.
- Data security was central to Cisco’s values. Pelican, our automated data validation tool, ensures data integrity and accurate validation. A production-like environment with static data facilitated effective testing.
- The teams developed comprehensive test strategies, including unit testing, System Integration Testing (SIT), and User Acceptance Testing (UAT). A robust testing framework was established, covering environment ownership, testing modes, entry criteria, and acceptance criteria. Detailed test cases were created to ensure thorough validation.
- Potential risks, such as data quality issues, data loss, system incompatibilities, and security concerns, were proactively identified and addressed. Methodical testing and detailed reporting ensured precision at every step. Continuous improvement was achieved through active engagement and feedback from stakeholders.
| Metric / Contribution | Before Pelican | Results After Pelican |
|---|---|---|
| Data Engineers Empowered | Limited access | 150+ data engineers empowered |
| Test Failure Rate & Auto-Generated Test Cases | Higher failure rate, no auto-generated test cases | 0% failure rate, 100% auto-generated test cases |
| Secure Validation | Data movement involved | No data movement |
| Triaging Efficiency & Testing Time | Lower efficiency, longer durations | 80% increase in efficiency, 50% reduction in testing time |
| Rows Validated | Limited validation | 2 trillion plus rows validated |
| Automated Validation | Manual validation | 100% Automated validation of data between one of the biggest on-prem legacy warehouse and BigQuery |
| Large-Scale Coverage | Limited | Validated 5,000 to 7,000 tables, over 1,000 validation jobs per day |
| Organization-Wide Usage | Limited | Used by over 100 non-QA developers |
| Extensive Mismatch Patterns | Limited sampling | Each validation covered more than 500 mismatch Patterns |
Results
By working in cohesion, Cisco and Onix achieved their goal of validating massive amounts of data with unparalleled efficiency. This project highlighted their technical prowess and the importance of collaboration and continuous learning in tackling complex data challenges.
This success not only met Cisco’s stringent requirements but also provided invaluable insights for the Onix team. The efficiency and accuracy demonstrated through this project have been pivotal in refining Onix’s services, allowing for further optimization of data migration and validation solutions across various industries.
