The client is a Europe-based e-commerce startup on a mission to empower small businesses and independent sellers by offering a highly customizable alternative to major platforms like Shopify. Their platform—spanning both web and mobile applications—gave users full control over the store’s admin panel and front-end, allowing store owners to tailor their shop’s look, feel, and functionality to their unique needs.
As the company gained traction, they faced an inevitable trade-off between rapid feature delivery and product stability. Without test automation, the increasing volume of updates led to frequent regressions, frustrated end-users, and mounting technical debt. That’s when our team stepped in to build a scalable and sustainable automation-first QA strategy—from the ground up.
Client requirements
When we joined the project, the client had no automation in place. Testing was performed entirely by a manual QA team, which quickly became a bottleneck as the product scaled and new features were shipped regularly. Regression checks were time-consuming and error-prone, and production issues were being discovered too late—often by users.
The client’s immediate goals were clear:
- Implement a test automation framework aligned with their AngularJS front-end stack
- Ensure rapid feedback during regression and smoke testing
- Reduce the QA cost footprint while maintaining confidence in releases
- Integrate automation into their existing CI/CD process (TeamCity)
- Improve visibility across test results with consistent, multi-channel reporting and collaboration tools
As the product matured, the requirements evolved to include:
- Migration from Protractor to WebdriverIO, ensuring modern tooling and long-term framework sustainability
- Adding mobile test coverage (iOS + Android) via Appium
- Incorporating API testing within the same codebase for full-stack validation
- Gradually replacing the manual team with a dedicated Automation QA team (AQA)
- Scaling test infrastructure to support parallel execution across 50+ sessions
Challenge
Taking a product from zero automation to a fully automated, cross-platform test ecosystem involved several complex phases, each with unique technical and process-driven hurdles.
Starting from Scratch
At the project’s inception, there were no automation assets, no frameworks, and no CI integration for testing. We were responsible for laying the entire foundation: selecting tools, designing the architecture, and aligning it with business workflows and delivery timelines.
Framework Migration: Protractor to WebdriverIO
Since the application was built with AngularJS, we initially used Protractor, which was purpose-built for Angular apps. However, as Protractor reached end-of-life and proved less stable under modern tooling expectations, we migrated the entire test suite to WebdriverIO. This shift improved performance, enabled cross-platform test support, and allowed the use of more flexible plugins and community tools.
Expansion to Mobile & API Testing
After the web testing pipeline stabilized, the client asked us to expand coverage to their mobile apps—spanning both iOS and Android. We introduced Appium and reused core test logic where possible using a modular test design. We also added API tests using the same repository and tooling, allowing full-stack testing under a unified architecture and seamless execution in CI.
Optimizing Business Logic Coverage
Interestingly, the initial test cases were written by the client, which meant they were already aligned with the core business logic and user workflows. We worked closely with their team to refactor and optimize these test cases—enhancing their structure for automation, eliminating redundancy, and filling coverage gaps.
QA Team Transition & Scaling
As automation matured, the client decided to transition entirely from manual QA to automation QA (AQA). This not only reduced costs but also improved feedback speed and overall product quality. We scaled the test suite and execution pipeline to run over 50 parallel sessions, ensuring regression cycles could run in minutes rather than hours.
Shared Environment & Complex Reporting
All testing occurred in a shared STAGING environment, requiring careful synchronization, stable test data, and collaboration across teams. Additionally, the client relied on several reporting tools—TestRail for manual case tracking, Allure Reports for visual results, Slack for alerts, and emails for stakeholders. We built a custom reporting layer to ensure consistent, reliable outputs across all systems.
Solution
We delivered a multi-platform, modular, and scalable automation framework tailored to the client’s current tech stack and future growth plans.
Initial Setup with Protractor
- Introduced test automation using Protractor, fully integrated with their AngularJS web app
- Focused on regression and smoke tests to quickly secure the most critical user flows
- Set up TeamCity pipelines to trigger tests on staging deployments and scheduled runs
Framework Migration & Cross-Platform Architecture
- Migrated framework to WebdriverIO, significantly improving maintainability and plugin support
- Refactored tests using Page Object Model and modular design patterns to reduce duplication
- Added Appium-based mobile automation for iOS and Android, reusing logic wherever possible
- Integrated API testing suites into the same codebase for consolidated CI execution
CI/CD & Test Execution Strategy
- Seamless integration with TeamCity, supporting:
- Triggered runs on staging deployments
- Scheduled overnight builds
- Smoke testing after production deployments
- On-demand runs by developers or stakeholders
- Parallelized execution using:
- BrowserStack for mobile tests across devices and OS versions
- High-resource internal nodes for fast web test execution
Collaboration, Reporting & Test Management
- Weekly syncs, shared documentation via Confluence, and proactive issue tracking
- Refined client-authored test cases into automation-ready specs
- Integrated Allure, TestRail, Slack, and email reporting via a custom reporting layer
- Ensured consistent visibility across QA, development, and management teams
Results & Business Impact
Over five years, our partnership with the client delivered remarkable outcomes that transformed their software quality, delivery speed, and QA strategy:
Quality & Confidence
- 90% reduction in production defects, thanks to thorough regression automation
- Greater confidence across stakeholders with each release—less firefighting, more focus on features
- By performing a full cycle of automated testing in 50+ threads, a side result was performing “load” testing on a daily basis, which helped to optimize and debug the resources required for optimal performance of the application
Speed & Efficiency
- 75% faster release certification process, enabling quicker time to market
- Regression cycles reduced from days to hours, running in 50+ parallel sessions
Cost Reduction
- Complete replacement of the manual QA team with automation specialists
- Lower long-term QA costs while increasing test coverage and consistency
Long-Term Growth Enablement
- A scalable, future-proof framework that supports Web, Mobile, and API in one codebase
- Strong collaboration culture established between devs, QA, and stakeholders
90%
reduction in production defects
75%
faster release certification process
50+
parrallel session
Tech details
- Frameworks: Protractor (initial), WebdriverIO (migrated), Appium for mobile
- Languages: JavaScript
- Test Types: UI (Web + Mobile), API, Smoke, Regression, Functional
- CI/CD: TeamCity
- Test Infrastructure: BrowserStack (mobile), Internal node for web, 50+ parallel test execution
- Architecture: Page Object Model, Modular Suite Organization, Cross-platform structure
- Reporting & Sync: Allure, TestRail, Slack, Email (via custom unified reporting pipeline)
- Test Source & Ownership: Business logic-based test cases authored by client, maintained and expanded by AQA
- Development Process: Waterfall (with proactive QA involvement during planning and deployment)
- Environments: Shared STAGING; targeted smoke checks on Production