Solution to Challenge 3 - Visual Testing with API Mocking

Challenge #3 is done — here's my solution to Challenge 3 - Visual Testing with API Mocking

Visual testing with playwright

I'm not a big fan of mock APIs for end-to-end tests, but for this specific challenge where the API changes every day and I need to ensure that the charts are displayed correctly, I think one approach is to mock the API always to return the same data and save the image snapshot to compare the current website against the saved snapshot.

Another challenge is that when you click on one segment of the 2 first two charts, a grid with the client's summary appears. On some charts, the developer can define the segments as individual elements, but on this chart there is only one canvas, and because the data is fixed, I created a function to click on a single segment by client name.

/**
* Click a pie/doughnut segment by its position in the dataset.
* Chart.js draws on <canvas> with no per-segment DOM node, so the click point is derived
* from the known dataset values (e.g. the mocked API response) plus the canvas' live
* bounding box, instead of a hardcoded pixel position. Assumes Chart.js defaults:
* segments start at 12 o'clock and are drawn clockwise in dataset order.
* @param values dataset values in the order they were rendered (same order as the mocked API response)
* @param index index of the segment to click
*/
async clickSegment(values: number[], index: number) {
     await test.step(`On "${this.locator.description()}" click segment #${index}`, async () => {
        await this.locator.scrollIntoViewIfNeeded();
        const box = await this.locator.boundingBox();
        if (!box) {
            throw new Error(`"${this.locator.description()}" has no bounding box (not visible?)`);
        }
        const total = values.reduce((sum, value) => sum + value, 0);
        const cumulative = values.slice(0, index).reduce((sum, value) => sum + value, 0);
        const midValue = cumulative + values[index] / 2;
        const midAngle = -Math.PI / 2 + (midValue / total) * 2 * Math.PI; // Chart.js doughnut: starts at 12 o'clock, clockwise
        const outerRadius = Math.min(box.width, box.height) / 2;
        const radius = outerRadius * 0.75; // midpoint of the doughnut ring (cutout ~50%-100%)
        const relativeX = box.width / 2 + radius * Math.cos(midAngle);
        const relativeY = box.height / 2 + radius * Math.sin(midAngle);
        await this.locator.click({ position: { x: relativeX, y: relativeY } });
    });
}

I created a visual helper to store the snapshots with 2 methods: one for the page and another for a specific element.

import { expect, Locator, Page, test } from '@playwright/test';

export class VisualHelper {

    constructor(private page: Page) {

    }

    /**
     * Check full page snapshot
     * @param snapshotName Snapshot name
     * @param timeout Max timeout
     * @param maxDiffPixels Max number of differing pixels allowed
     */
    async checkPageSnapshot(snapshotName: string, timeout = 5_000, maxDiffPixels = 100) {
        const stepDescription = 'Compare snapshot: ' + snapshotName + ' with maxDiffPixels: ' + maxDiffPixels;
        await test.step(stepDescription, async () => {
            await expect(this.page).toHaveScreenshot(snapshotName, {
                timeout: timeout,
                maxDiffPixels: maxDiffPixels
            });
        });
    }

    /**
     * Check element snapshot
     * @param element Element to check the snapshot
     * @param snapshotName Name of the snapshot
     * @param timeout Max timeout
     * @param maxDiffPixels Max number of differing pixels allowed
     */
    async checkElementSnapshot(element: Locator, snapshotName: string, timeout = 5_000, maxDiffPixels = 100) {
        const stepDescription = 'Compare snapshot: ' + snapshotName + ' with maxDiffPixels: ' + maxDiffPixels;
        await test.step(stepDescription, async () => {
            await expect(element).toHaveScreenshot(snapshotName, {
                timeout: timeout,
                maxDiffPixels: maxDiffPixels
            });
        });
    }
}

To mock the API I created API with the fixed data, for example:

{
    "Amount": 56046.00,
    "AverageDaysPastDue": 35
}

To mock the API I added this code

/**
* Mock an api
* @param description Description for the HTML reporter
* @param url API URL to mock
* @param jsonData JSON that will be returned
* @param status HTTP status code to return (defaults to 200)
*/
async mockApi(description: string, url: string, jsonData: any, status = 200) {
    await test.step(description, async () => {
        await this.page.route(`**${url}`, async route => {
            await route.fulfill({
                status: status,
                contentType: 'application/json',
                body: JSON.stringify(jsonData),
            });
        });
    });
}

For mock the summary I created this function on the DashboarPage.ts

async mockSummary() {
    const stepDescription = 'Modify the summary with fixed data';
    await test.step(stepDescription, async () => {
        await this.apiHelper.mockApi(
            stepDescription,
            '/api/collection/summary',
            summary,
        );
    });
}

The most basic example is:

import { expect, test } from '../fixtures';
import { DashboardPage } from '../pages/DashboardPage';
import top5Delay from '../data/mocks/top5Delay.json';
import top5Total from '../data/mocks/top5Total.json';
import top10Limit1 from '../data/mocks/top10Limit1.json';

test.describe('Check Dashboard', () => {
    let dashboardPage: DashboardPage;
    test.use({ storageState: '.auth/admin.json' });

    test.beforeEach(async ({ page, locale }) => {
        dashboardPage = new DashboardPage(page, locale);
        await dashboardPage.mockAllApis();
        await dashboardPage.goTo();
        await dashboardPage.waitForChartsAreVisible();
    });

    // eslint-disable-next-line playwright/expect-expect
    test('Should show dashboard', {
        tag: ['@VisualTesting', '@Dashboard'],
    }, async () => {
        await dashboardPage.checkPageSnapshot();
    });

I made some changes to the dashboard. In the past, there was only one row that displayed the data of the currently selected segment, but to improve the chart's accessibility for people with low vision, I added it to a table, and visual testing detected that change.

With this approach, if there is a minor change to the charts, you will see an error, adding in red the pixels that are different:

This was the expected snapshot

Actual snapshot

To update the saved snapshot you can execute this command:

npx playwright test --update-snapshots

You can check my previous article: Visual Testing with Playwright

Unit Testing with in memory database

For my .NET API project, I created unit tests using an in-memory database. With this approach, I can test with an empty database and create the necessary information to test each API and check the results against known data.

var options = new DbContextOptionsBuilder<MicrosipContext>()
            .UseInMemoryDatabase(databaseName: Guid.NewGuid().ToString())
            .Options;
        using var isolatedContext = new MicrosipContext(options);

To add data to test I created a function addInvoice

    private static void AddInvoice(DBContext context, int invoiceId, int clientId, decimal amount, DateTime dueDate)
    {
        context.Invoice.Add(new Invoice
        {
            Id = invoiceId,
            ClientId = clientId,
            Number = $"V{invoiceId:00000000}",
            Date = DateTime.Now,
            Cancel = "N",
            Amount = amount,
            DueDate = dueDate
        });
    }

The unit test will be:

[Fact]
public async Task GetTotalByClient_WithTwoInvoices_AggregatesTotalAndAverageOverdueDays()
{
    // Own in-memory database, independent of the shared MemmoryContext fixture and of
    // GetDetailed/GetTotalByInvoiceAsync's own tests, so this test's expected values come
    // from hand-picked numbers rather than from another DAO method's output.
    var options = new DbContextOptionsBuilder<MicrosipContext>()
        .UseInMemoryDatabase(databaseName: Guid.NewGuid().ToString())
        .Options;
    using var isolatedContext = new MicrosipContext(options);

    const int clientId = 700;
    isolatedContext.Clientes.Add(new Clientes { ClienteId = clientId, Nombre = "Test Client", Estatus = "A", SujetoIeps = "N" });

    const decimal invoice1Amount = 1000m;
    var invoice1DueDate = DateTime.Now.AddDays(-30);
    const decimal invoice2Amount = 2000m;
    var invoice2DueDate = DateTime.Now.AddDays(-10);

    AddInvoice(isolatedContext, invoiceId: 1, clientId: clientId, amount: invoice1Amount, dueDate: invoice1DueDate);
    AddInvoice(isolatedContext, invoiceId: 2, clientId: clientId, amount: invoice2Amount, dueDate: invoice2DueDate);
    isolatedContext.SaveChanges();

    var collectionDAO = new CollectionDAO(isolatedContext, new ParameterMock());

    var totals = await collectionDAO.GetTotalByClientAsync();

    var expectedTotal = invoice1Amount + invoice2Amount;
    var overdueDays1 = (DateTime.Now.AddDays(1) - invoice1DueDate).Days;
    var overdueDays2 = (DateTime.Now.AddDays(1) - invoice2DueDate).Days;
    var expectedAverageDaysOverdue = (int)Math.Round((overdueDays1 + overdueDays2) / 2.0, MidpointRounding.AwayFromZero);

    var clientTotal = Assert.Single(totals);
    Assert.Equal(clientId, clientTotal.ClientId);
    Assert.Equal(expectedTotal, clientTotal.Total);
    Assert.Equal(expectedAverageDaysOverdue, clientTotal.DaysOverdue);
}

Another option to integration tests is using a TestContainer an generate the data in a Docker database. You can read about this:
Testcontainers Best Practices for .NET Integration Testing

API testing

Because the validation was already on unit test I didn't want to duplicate the logic in another test that can execute the API and with a SQL query compare with db query. I only test that the API returns correct status code and check the response body against a schema.

Postman API testing

One popular library to validate schemas is Zod you can use in Postman but that option is only available on Postman CLI is not available on postman newman.

This is one example to validate the json schema with Zod:

const { z } = pm.require('npm:zod');

const itemSchema = z.object({
    ClientId: z.number(),
    Client: z.string(),
    Total: z.number(),
    DaysOverdue: z.number()
});

I created a global function to validate the Zod schema

CheckArrayZodSchema: function(itemSchema) {
    const { z } = pm.require('npm:zod');
    // Wrap in z.array so the whole response array is validated
    const responseSchema = z.array(itemSchema);

    const response = pm.response.json();

    // Validate data safely
    const validation = responseSchema.safeParse(response);

    pm.test("Response matches Zod schema", () => {
        pm.expect(validation.success).to.be.true;
    });

    // Log helpful details if validation fails
    if (!validation.success) {
        console.error("Zod Validation Failures:", JSON.stringify(validation.error.format(), null, 2));
    }
  }

Postman also includes an option to visualize the results as a chart. You can request the AI integrated to this. The code generated is:

var template = `
<canvas id="myChart" height="100"></canvas>
<script src="https://cdnjs.cloudflare.com/ajax/libs/Chart.js/2.5.0/Chart.min.js"></script> 
<script>
var ctx = document.getElementById('myChart');
pm.getData(function (err, value) {
    var myChart = new Chart(ctx, {
        type: 'pie',
        data: {
            labels: value.response.map(item => item.Client),
            datasets: [{ 
                label: [],
                backgroundColor: ["#003f5c", "#58508d", "#bc5090", "#ff6361", "#ffa600"],
                borderWidth: 1,
                data: value.response.map(item => item.DaysOverdue)
            }]
        },
        options: {
            legend: { display: true },
            title: {
                display: true,
                text: 'Top 5 Delay'
            }
        }
    });
});
</script>`;

pm.visualizer.set(template, {
    response: pm.response.json()
});

You can visualize the chart on the Visualization Tab

RestAssured

For RestAssured I only validate agains the schema

[Test]
public async Task GetSummary_WithValidUser_ReturnsGlobalSummary()
{
    var response = Given()
          .Header("Authorization", $"Bearer {AuthToken}")
        .When()
          .Get($"{BaseUrl}/{CollectionEndpoint}/summary")
        .Then()
          .StatusCode(200)
        .DeserializeTo<CollectionSummary>();
    await Assert.That(response!.Amount).IsGreaterThanOrEqualTo(0);
    await Assert.That(response.AverageDaysPastDue).IsGreaterThanOrEqualTo(0);
    }

RestSharp

I did the same on RestSharp:

[Test]
public async Task GetSummary_WithValidUser_ReturnsGlobalSummary()
{
    var client = ApiClient.Create(Configuration, AuthToken);
    var response = await client.GetAsync<CollectionSummary>($"{CollectionEndpoint}/summary");

    await Assert.That(response.StatusCode).IsEqualTo(200);
    await Assert.That(response.Data!.Amount).IsGreaterThanOrEqualTo(0);
    await Assert.That(response.Data.AverageDaysPastDue).IsGreaterThanOrEqualTo(0);
    }

You can check check my solution on Testing Dojo repository

Thanks for following along these challenges. Testing is all about continuous learning, so don't hesitate to ask questions or share your feedback below.

If this helped you in any way, feel free to share it with the community.