Cypress timeout – uncovering website performance issues

 We were working on building a site that was a bridge between interested customers and the car dealers/OEMs. All the data related to a vehicle which will be useful for a customer was displayed on the website. The data from interested buyers was also captured through a form and shared with the relevant dealers/OEMs.

As the project was growing day by day, new features were getting added. So, manual regression testing was becoming a lengthy process before every release which happened twice a week.

The website was on ReactJS. We were using Cypress for automation because it is purely based on JavaScript and uses a unique DOM manipulation technique and operates directly in the browser. Cypress also provides a unique interactive test runner in which it executes all commands. We started automating the regression test cases according to the priority of the module. The regression suite was integrated with CI. Before every release, it was mandatory for the regression suite to pass in the CI.

Problem Statement 

There were two releases in a week, which meant that new features were added every time. Suddenly we started facing issues on CI and test cases started failing randomly with timeout errors. 

We tried running it multiple times but test cases that were passed earlier start failing, on every run we were seeing various sets of test cases failing which were passing before, the result was not the same in every run. 

We suspected that there was some issue with the Cypress tests, so we tried to replicate the issue on our local machine. But many test cases randomly started giving timeout errors on our local machines as well.

What is a timeout?

All the commands in Cypress have a default timeout that means Cypress will wait for a definite time for the command to get executed, all the DOM-based commands have a default timeout of 4 seconds.

For example cy.get(‘#firstname’), in this case Cypress waits up to 4 seconds for id to be present in DOM. Similarly for every assert command Cypress waits up to 4 seconds by default. It looks for the element up to 4 seconds and if it is not present, Cypress throws an error 'Timed out retrying after 4000ms: Expected to find element: #firstname, but never found it'.

So the conclusion was that our Cypress test suite failed due to the timeout issues. When testing a web application, a timeout error can occur when the application performs an asynchronous operation that needs to finish before the application state and user interface (UI) are ready for testing. If a command or assertion in Cypress is executed before this operation completes, the test will mostly fail.

However, if the time taken by this operation varies, sometimes it may complete quickly enough to pass the test. This inconsistency creates a situation where the test results become unreliable or "flaky."

Our Solutions 

On looking at the timeout errors, our first approach was to increase the default timeout to 20 seconds, just to check if the test cases started passing. Yes, all the test cases started passing after we increased the default timeout.

How did we decide the timeout for our application?

Now, the next question was what should be the value of default timeout? We thought of checking the performance of our website. We found that the average time taken by all the pages to be interactive was around 8.5 seconds. 

When we increased the default timeout to 10 seconds in cypress.json our test cases started working on CI.

1. Specifying timeouts: 

We can modify the default timeout for a single command by passing a timeout parameter, for the example in this contact form , if we want to add a 10 seconds timeout we can modify it as below:

it('Contact us form submission',()=>{
       	cy.get('.fullname', { timeout: 10000 }).type('Name XYZ');
      	cy.get('.email', { timeout: 2000 }).type('abcd@abc.com');
      	 cy.get('.enquiry', { timeout: 2000 }).type('Text message enquiry');
      	 cy.get('.contact-us-button', { timeout: 2000 }).click();
      	 cy.get('.result',{ timeout: 10000 }).should('be.visible');  
 	  })

‍
Here in the above code we have given different timeouts for each command, so cypress will wait for the wait for a given time (only if an element is not found) before moving to the next command. 
We can also globally configure by setting these timeouts in cypress,json which will by default add a 10-second timeout for all the commands.
 

"defaultCommandTimeout": 5000,

2. cy.wait() function

Lets understand cy.wait() , this will wait for specified milliseconds or wait for an aliased resource to complete to move forward . Here we can specify the time period. If needed this is simple way to specify wait time Example for 5 seconds
 

cy.get('.button').click();
      	 // Wait for a specific period of time (e.g., 5 seconds)
       	cy.wait(5000);
       	// Continue with other assertions or actions after the wait
       	cy.get('.result').should('be.visible');

‍

1. Request timeout : Default Request Timeout is 5000 ms . This means that cypress by default will wait for 5000ms for a matching request to leave the browser . If no matching request is found , cypress will throw an error.
2. Response timeout : This is the second wait period which is 30 secs by default. After cypress affirms matching requests being sent, it waits for the response for this request . if there is no response from the external server , an error is thrown.

cy.get('.contact-us-button').click();
      	 cy.get('.result').invoke('text').as('resultText');
 // Wait for the alias to have a specific value
     	 cy.wait('@resultText').should('eq', 'Your enquiry has been successfully sent to the store owner');
 // Continue with other assertions or actions after the wait
      	 cy.get('.topic-block-body').should('be.visible');
      	 })
       	})

‍

In the example above:

• We first perform an action, such as clicking a button, which triggers an asynchronous operation.
• Then, we use .invoke('text').as('resultText') to assign the text value of an element with the class .result to the alias resultText.
• We use cy.wait('@resultText') to wait for the alias resultText to have a specific value. In this case, we expect it to be 'Expected Value'.
• Once the alias value matches the expected value, the test proceeds to the next step, where we can continue with other assertions or actions.

By using cy.wait() with an alias, you can ensure that Cypress waits for a specific condition before moving on to the next steps in your test. This is useful when dealing with asynchronous operations and waiting for certain values to be available or updated.

Conclusion

We were continuously adding new features to the site, due to which performance of the site was impacted, and our Cypress tests started failing because the time required for rendering the elements on the pages was high. The real issue was with the performance of the page. These reports were shared with the development team and it was decided to improve the performance of the websites in the next sprints. 

It's not always that your test needs to be fixed. We need to identify the root cause and provide feedback to the team.