Introduction
As a test automation architect with over a decade of experience spanning thousands of real-world devices, properly handling errors is a critical skill I‘ve honed across projects of all sizes. And whether you‘re just beginning with Cypress or an advanced practitioner, you‘ll inevitably encounter your fair share of exceptions.
In my time eliminating flakes and building reliable automation, I‘ve seen firsthand how unhandled errors lead to fruitless debugging, undetected regressions, and ultimately erosion of trust in our test suites. But with the right strategies and Cypress techniques, we can surface and handle errors with grace – preventing flakes without masking issues.
In this complete guide, I‘ll share the lessons learned over my career to help you become a error handling master. We‘ll cover:
- Common exception types in Cypress
- BEST PRACTICES for handling errors
- How to track and analyze errors over time
- Automation opportunities for ease of maintenance
Let‘s get started conquering errors for stable, resilient test suites!
Error Handling – An Industry Pain Point
Modern web applications are complex beasts – issues like browser inconsistencies, network interrupts, undefined behavior in source code – all increase likelihood of runtime exceptions. In fact, by some estimates up to 60% of JavaScript based applications contain unhandled exceptions [Citation].
And the impacts cascade through our testing:
- Flaky, unreliable tests erode confidence
- Errors mask real app regressions when blindly handled
- Spread error handling logic causes maintenance burdens
Cypress‘s error first architecture surfaces these issues reliably across test runs for early detection. Now it‘s our responsibility as test authors to handle them properly through validation, retries, analytics, and more.
While no one enjoys encountering errors (us developers least of all!), arming ourselves with the techniques in this guide makes them far less intimidating. Let‘s get started!
Error Types in Cypress
Before we can handle errors effectively, we need to level-set on the categories we‘ll encounter:
| Error Type | Origin | Example |
|---|---|---|
| Application Under Test | Issues within the app itself | JavaScript exceptions, console errors |
| Test Code | Problems in Cypress test code | Incorrect selectors, assertions |
| Status Codes | Non-200 HTTP codes | 400 Bad Request, 500 errors |
As we explore handling techniques, recognizing the error type will inform our strategy. Now let‘s tackle them one by one!
Handling Errors from Application Under Test
Runtime errors within the web app itself – outside test code – are a common culprit of flakes. Maybe a certain browser triggers a JavaScript syntax errors, or the network glitches leading to stalled requests. Left unaddressed, these types of errors fail tests despite no actual regressions.
Some examples include:
- JavaScript Syntax Errors: Uncaught syntax issues like invalid tokens, identifiers
- Console Errors: Warnings and exceptions surfaced to console
- CORS/Network Issues: Resources blocked due to access policy errors
To handle these, Cypress provides the cy.on() method:
cy.on(‘uncaught:exception‘, (err) => {
// inspect error instance
// return false to prevent test failure
})
We can check the error message to selectively handle known exceptions:
cy.on(‘uncaught:exception‘, (err) => {
if(err.message.includes(‘Unexpected Token‘)){
// handle syntax error
return false
}
// throw other errors
})
Blanket handling without inspection can prevent detecting unintended app regressions. Choose wisely which errors to handle vs allow to fail tests.
Real-World Examples
Let‘s walk through some common real-world cases:
JavaScript Syntax Errors
Certain syntax errors like invalid tokens only appear in specific browser and OS combinations.
Uncaught SyntaxError: Unexpected token ‘<‘
We can safely auto-handle these known errors:
cy.on(‘uncaught:exception‘, (err) => {
if(err.message.includes(‘Unexpected Token‘)){
return false
}
})
No need to fail tests for issues unrelated to changes.
Console Errors
Apps may log non-critical warnings to console that throw in Cypress.
Warning: Component X is deprecated
We can silence warnings about known tech debt:
cy.on(‘uncaught:exception‘, (err) => {
if(err.message.includes(‘Component X is deprecated‘)){
return false
}
})
This prevents failures while teams resolve the underlying console logs.
Handling Test Code Errors
Even with robust test practices, it‘s inevitable bugs in our automation code will arise – flaky locators, race conditions, test data issues. Without proper handling, minor test defects can block debugging cycles and mask application problems.
Some examples:
- Flaky Element Lookups: Timeout errors when elements intermittently not located
- Race Conditions: Async timing issues around test data creation/clean up
- Test Data Failures: Network issues preventing test data setup via API
Cypress gives us Cypress.on() to handle fails from our automation code:
Cypress.on(‘fail‘, (error) => {
// inspect error
// retry or return to prevent failure
})
This allows us to retry element lookups if they intermittently fail:
Cypress.on(‘fail‘, (error) => {
if(error.message.includes(‘timed out‘)){
cy.get(‘@element‘).should(‘be.visible‘) //retry
} else {
throw error // throw other errors
}
})
Tip: Apply this pattern judiciously to avoid masking legitimate regressions!
Real-World Examples
Some practical instances where we might handle Cypress test code errors:
Flaky Element Lookups
Timed out retrying: failed to find #button
This could happen when the page loads slowly in certain environments. We can retry to prevent flakiness:
Cypress.on(‘fail‘, (error) => {
if(error.message.includes(‘#button‘)) {
cy.get(‘#button‘).should(‘be.visible‘)
}
})
But avoid over-handling without debugging root cause!
Test Data Creation Failures
If API requests to set up test data fail, that can block unrelated tests. We could handle to reduce wasted developer time debugging false failures:
Cypress.on(‘fail‘, (error) => {
if(error.message.includes(‘POST /test-data‘)) {
// log and alert team to fix
return
}
})
Retrying requests could help depending on root cause as well.
Automating Error Handling
Error handling logic can grow redundant across tests. Centralizing reusable logic reduces duplication.
Some options:
- Mocha Hooks: before/after hooks to catch errors
- Custom Commands: Common actions wrapped with handling
For example, handling application errors globally:
beforeEach(() => {
cy.on(‘uncaught:exception‘, (err) => { // logic return false })
})
Or retry commands on intermittent failures:
Cypress.Commands.add(‘retryOnFail‘, { (el) => {
Cypress.on(‘fail‘, (error) => { cy.get(el) })
return cy.get(el)
}
})
This reduces duplication and makes maintenance simpler long term.
Tracking Errors Over Time
Reviewing error metrics and patterns gives us signal into potential regressions. Using Cypress Dashboard, we can surface trends including:
- Total tests failing due to errors over builds
- Frequency distribution of various errors
- Errors by browser, operating system, environment
Monitoring for systemic vs intermittent errors guides our handling strategy. Do we need to handle universally or just in certain environments? Iterating based on analytics prevents future test debt.
Best Practices
While error handling prevents flaky tests, balancing real issue detection is key. Keep these tips in mind:
- Don‘t blindly handle all errors: Carefully validate and selectively allow failures only from known safe exceptions.
- Analyze over time: Enrich analytics with error message data, tally frequent errors for proactive handling.
- Debug true root cause: Understand the core issue vs just masking symptoms to prevent larger debt issues.
By keeping error handling focused, we gain test stability without losing critical signal into regressions.
Key Takeaways
Robust error handling is a keystone habit on the path to being a test automation pro. Cypress gives us powerful callbacks and customization options to implement smart handling strategies.
No one enjoys encountering errors, but arming ourselves with the right techniques makes them infinitely less disruptive. Just remember:
- **Validate errors selectively**: Don‘t blindly handle all errors
- **Customize handling strategies**: Based on error types
- **Analyze over time**: To prevent future regressions
- **Automate and centralize**: Minimize duplication
By mastering these error handling foundations early on with Cypress, you‘ll prevent countless hours lost to debugging flakes and stay laser focused on building quality into your apps.
Now no error stands a chance against you! 💪
