How To Fix Flaky Tests in Rendering and Animation Workflows

UI rendering and animations are what give an app a great and interactive user experience. As interesting as this concept is, so is how frustrating it can become if things go wrong. And one way that could happen is tests being flaky.

There are actually multiple reasons for flaky tests in rendering or animations, but one of the main reasons is timing.

For instance, you might have a spring animation of a button displaying. Now testing this might result in flakiness, all because the animation finished some milliseconds too late due to a random background running process.

In this guide, we’ll be using Jest and React testing libraries to describe real-world examples of tests for better understanding. However, you can follow along, irrespective of your stack.

Root causes

Flaky tests, in general, have quite a number of common causes. However, when being specific to UI rendering and animations, here are the main root causes:

Timing Issues

Animations by default often come with setTimeout delays or transition-duration properties. These timings are sometimes what make assertions fail resulting in flakiness.

Take a look at this test for a component with an animation whereby its width gets increased with the click of a button:

test("Box width increases", () => {
  render(<Box />);
  const box = screen.getByTestId("box");
  expect(box).toHaveStyle({ width: "70px" });

  fireEvent.click(screen.getByRole("button"));
  expect(box).toHaveStyle({ width: "100px" });
});

The problem right here is that after clicking the button to increase the width, the assertion might execute before the animation of the width increase finishes depending on any timing events.

Thus, the main challenge in rendering and animation testing comes down to making sure that assertions are made after the animation has fully executed and not earlier. A nice approach would be to wait for the animation to complete; in React testing, this can be achieved with the use of waitFor.

Environmental Dependencies

The system resources come into play in this case. In a way, tests created for rendering complex animations can be dependent on system resources like CPU or memory.

Now this won’t be an issue for machines that are capable. However, if the machine has limited resources, animation rendering could be slow and less smooth, thus triggering random test failures.

Another instance of environmental dependencies can be network connectivity. Let’s say you have an animation that relies on fetching external assets; this alone introduces unexpected factors as network fluctuations can influence the animation behavior. A way to fix this would be with the use of mocks.

Libraries Dependencies

Often times, animations used are from libraries, as it is much faster and more efficient than writing the code from scratch.

In React for instance, there are tons of animation libraries like Framer Motion, React Spring, etc. However, these libraries could cause flaky tests when:

• the version of the third-party library and the current version of the technology stack used are not compatible. This case is sometimes silent, as errors might not be displayed, just that the tests results seems to be unstable.
• the animation libraries used have dependencies that interfere with one another.
• the animation libraries have internal bugs that haven’t been fixed by the library developers which only show up during testing.
• developers use animation libraries in a way it wasn’t designed for. Interestingly enough, the library might perform the task as expected but since it was used incorrectly, testing can provide inconsistent results.

Fixing Flaky Tests in Rendering and Animation

Now, to make this guide more elaborate and easier to understand, we’ll provide four real-world code examples and show the wrong way causing flaky tests and the right way to fix them.

Rendering of a Newsletter Prompt

In this example, this component renders a simple form with an email input and a subscribe button. When submitted, it logs the email and resets the state.

function NewsletterPrompt() {
  const [email, setEmail] = useState("");
  const [subscribed, setSubscribed] = useState(false);

  const handleSubmit = (e) => {
    e.preventDefault();
    setSubscribed(true);
  };

  return (
    <div>
      {!subscribed ? (
        <div>
          <h2>Subscribe to Newsletter</h2>
          <input
            type="email"
            placeholder="Email address..."
            value={email}
            onChange={(e) => setEmail(e.target.value)}
          />
          <button type="submit" onClick={handleSubmit}>
            Subscribe
          </button>
        </div>
      ) : (
        <p>Subscribed successfully</p>
      )}
    </div>
  );
}

Now let’s write a test for the component that’ll show how flakiness can happen:

test("Newsletter prompt renders and can be subscribed to", () => {
  render(<NewsletterPrompt />);
  const emailInput = screen.getByPlaceholderText("Email address...");
  const subscribeButton = screen.getByText("Subscribe");

  fireEvent.change(emailInput, { target: { value: "xyz@example.com" } });
  fireEvent.click(subscribeButton);

  expect(screen.getByText("Subscribed successfully")).toBeInTheDocument();
});

The problem here is that making assertions directly like this could run before the component even re-renders to show updates of the submitted email.

This test is bound to be flaky because it relies on the component being rendered synchronously, which might not always be the case. As rendering can sometimes be asynchronous.

A fix for this test would be something like this:

test("Newsletter prompt renders and can be subscribed to", async () => {
  render(<NewsletterPrompt />);
  const emailInput = screen.getByPlaceholderText("Email address...");
  const subscribeButton = screen.getByText("Subscribe");

  fireEvent.change(emailInput, { target: { value: "xyz@example.com" } });
  fireEvent.click(subscribeButton);

  expect(
    await screen.findByText("Subscribed successfully"),
  ).toBeInTheDocument();
});

In this fix, we used async/await and the screen.findByText method to expect and wait for the text content element to show up in the DOM, which would prove that the component has indeed re-rendered with the “Subscribed successfully” message.

An Animated Button

Let’s say you have a button that animates its color from orange to blue when clicked and reverts to orange when clicked again.

export default function AnimatedButton() {
  const [isAnimated, setIsAnimated] = useState(false);

  const handleClick = () => {
    setIsAnimated(!isAnimated);
  };

  const buttonStyle = {
    width: "100px",
    height: "50px",
    backgroundColor: isAnimated ? "blue" : "orange",
    transition: "backgroundColor 0.5s ease",
  };

  return (
    <button style={buttonStyle} onClick={handleClick}>
      Animate
    </button>
  );
}

Now here’s a common but incorrect way to write the test for this button color change, as it could show flaky behavior:

import { render, fireEvent } from "@testing-library/react";
import AnimatedButton from "./AnimatedButton";

test("background color changes", () => {
  render(<AnimatedButton />);
  const button = screen.getByRole("button", { name: "Animate" });

  fireEvent.click(button);
  expect(button).toHaveStyle({ backgroundColor: "blue" });

  fireEvent.click(button);
  expect(button).toHaveStyle({ backgroundColor: "orange" });
});

This test looks really straightforward; it renders the button, clicks it, and then asserts the expected color changes. However, there’s a bit of a weak spot, which is where flakiness comes in.

The reason this test might sometimes show flaky results is that after each click event, it assumes that the animation completes immediately, as we aren’t considering the timing of the animation before asserting the state of the button’s background color.

Instead, this would be the right way:

test("background color changes", async () => {
  render(<AnimatedButton />);
  const button = screen.getByRole("button", { name: "Animate" });

  fireEvent.click(button);
  await waitFor(() => expect(button).toHaveStyle({ backgroundColor: "blue" }));

  fireEvent.click(button);
  await waitFor(() =>
    expect(button).toHaveStyle({ backgroundColor: "orange" }),
  );
});

With this simple change of using waitFor, we can be certain that the test would allow the animation to be completed before asserting the background color of the button. This makes sure that the test assertion is in sync with the UI’s rendering update.

Rendering of a Basic Modal

For this example, let’s use a simple modal component that when a button is clicked, the visibility of the modal changes:

function AnimatedModal() {
  const [isVisible, setIsVisible] = useState(false);
  const toggleVisibility = () => setIsVisible(!isVisible);

  return (
    <div>
      <button onClick={toggleVisibility}>Toggle Modal</button>
      {isVisible && <div data-testid="animated-modal">Animated Modal</div>}
    </div>
  );
}

Now as simple as this component is, testing if it renders correctly can become a hassle because of the animation.

Here is a wrong way to write the test:

test("Toggling animated modal", () => {
  render(<AnimatedModal />);
  const toggleButton = screen.getByRole("button", { name: "Toggle Modal" });

  expect(screen.queryByTestId("animated-modal")).not.toBeInTheDocument();
  fireEvent.click(toggleButton);
  expect(screen.getByTestId("animated-modal")).toBeInTheDocument();
});

As we explained in the previous example, this test also asserts the presence of the modal content immediately after the click event. So due to the animations being asynchronous, this test is bound to be a flaky one.

Here’s the right way:

test("Toggling animated modal", async () => {
  render(<AnimatedModal />);
  const toggleButton = screen.getByRole("button", { name: "Toggle Modal" });
  expect(screen.queryByTestId("animated-modal")).not.toBeInTheDocument();

  fireEvent.click(toggleButton);
  await waitFor(() => {
    expect(screen.queryByTestId("animated-modal")).toBeVisible();
  });

  fireEvent.click(toggleButton);
  await waitFor(() => {
    expect(screen.queryByTestId("animated-modal")).not.toBeInTheDocument();
  });
});

In this fix, we used waitFor to assert the modal’s presence or absence visibility only after the animation has been completed. This gives us the certainty that the test is more reliable.

Strategies to Reduce Flaky Tests in Rendering and Animation

So when writing tests specifically for UI rendering and animations, here are some things to look out for or what you should use instead to reduce the chance of flaky tests showing up:

• Always review the test logic written as it is one of the common mistakes made as explained in the section above.
• Network delays or fluctuations should be accounted for and mock functions can come in handy in fixing this.
• When testing on UI rendering, avoid using DOM selectors like .querySelector() to get elements, instead use the queries provided by the testing library like getBy, queryBy, or findBy.
• Ensure that the version of the animation library used is compatible with the version of the tech stack used.
• Keep the rendering or animation tests in smaller units i.e. try to only test one component at a time. This is more efficient and easier to debug flakiness.
• Know when to use the appropriate assertion matchers, for instance when testing animations, toBeVisible() is better suited than toBeInTheDocument().
• Use waitFor for assertions and not for firing events.
• You can use jest.useFakeTimers() as an alternative to waitFor for further control of testing.
• Some developers skip/disable animations when testing, while it’s not recommended, it is an option.

Conclusion

Flaky tests in rendering and animations can actually be difficult to deal with but with a proper understanding of the common causes specifically timing complications, the flakiness of tests can be reduced tremendously.

For further reading on flaky tests, check out these resources:

• Flaky Tests In React: Detection, Prevention and Tools
• Best Practices for Identifying and Mitigating Flaky Tests
• How to Avoid Flaky Tests in Playwright