Migrating a Production Django App from Elasticsearch to OpenSearch

Originally published on HackerNoon.

Key Takeaways

• Legacy search clients like elasticsearch==7.10.1 hard-cap underlying transport libraries (urllib3<2), blocking critical security patches for vulnerabilities like CVE-2025-50181.
• elasticsearch-py 7.14.0+ introduced a client-side UnsupportedProductError check that refuses connections to non-Elasticsearch servers, making opensearch-py the necessary drop-in for AWS OpenSearch domains.
• Upgrading to opensearch-py alone does not guarantee a security fix because its urllib3 constraint is a floor (>=1.26.19), requiring an explicit application-level pin of urllib3>=2.5.0.
• When third-party packages like django-elasticsearch-dsl-drf are abandoned, vendoring a lean (~190 LOC) project-local filter and pagination layer prevents breaking client-facing REST API response contracts.
• Verifying a search client migration requires separating metadata ping traffic (size=0) from document retrieval queries and verifying vulnerability remediation with automated security auditing tools.

When a security advisory hits a core library like urllib3, upgrading is usually a routine dependency bump. However, when legacy search clients hard-cap underlying transport libraries, security remediations become full-scale client migrations.

This is the story of that digging: how a security patch that looked like a one-line version bump turned into a full client-library migration, what part of that migration was genuinely mechanical, and what part wasn't. If you're running an older elasticsearch Python client against an AWS OpenSearch domain, there's a decent chance you're sitting on the same trap.

The Anatomy of a Transitive CVE Trap

In modern Python backend architectures, transitive dependencies can quietly turn into security bottlenecks. Our production Django application relied on elasticsearch==7.10.1 to interact with event and audit-log indices hosted on AWS OpenSearch 1.3.

A transitive dependency is a package your code never imports directly, it gets pulled in because something you depend on needs it. That distance is exactly why a cap like this goes unnoticed for years: nobody on the team ever wrote import urllib3, so nobody was watching its version.

When CVE-2025-50181 (an open-redirect / SSRF vulnerability in urllib3, affecting everything before 2.5.0, details here) was disclosed, security compliance required updating urllib3 to version 2.5.0 or higher. However, running a dependency audit revealed a hard blocker:

# pyproject.toml (legacy state)
elasticsearch = "7.10.1" # Pinned: requires urllib3 >= 1.21.1, < 2

The elasticsearch 7.10.1 package explicitly capped urllib3 < 2. Because 1.26.20 was the final release on the urllib3 1.26.x line, no backport was available. Upgrading elasticsearch-py to 7.14+ or 8.x was not an option due to the introduction of client-side product checks (UnsupportedProductError), which intentionally refuse connection to AWS OpenSearch or Elasticsearch OSS endpoints.

The client library itself was forcing the application to remain vulnerable. The only remediation path was swapping the underlying search client entirely.

Client Selection: Why opensearch-py, Not a Newer Elasticsearch Client

OpenSearch is AWS's fork of Elasticsearch: a separately maintained, Apache-licensed search engine with (mostly) the same API, created specifically so companies could keep running an open-source-licensed alternative after Elastic changed its own license.

The elasticsearch==7.10.1 pin predates a fight that had nothing to do with us. Elastic changed its licensing in January 2021 after AWS forked Elasticsearch into what became OpenSearch, and that July elasticsearch-py 7.14.0 quietly added a check that raises UnsupportedProductError on any server missing Elastic's own X-Elastic-Product header (issue #1639). AWS OpenSearch doesn't send it. Elastic called the change an enhancement; the backlash was loud enough that Elastic locked the GitHub thread. Practically, it meant any elasticsearch-py release from 7.14 onward refuses to talk to our production database, so upgrading the vulnerable client in place was never on the table.

opensearch-py sidesteps that fight by construction: it's OpenSearch's own client, forked from elasticsearch-py at the 7.10.x line before the check existed. It never inherited it, and from a recent enough release, it drops the urllib3 ceiling too. COMPATIBILITY.md confirms it supports OpenSearch 1.0 through 2.x, our production range, and since OpenSearch's wire protocol is still the Elasticsearch 7.x REST API, the same client talks cleanly to both our production domain and the legacy elasticsearch-oss:7.10.2 container we ran in CI.

Picking a Version Was Its Own Research Project

Here's the part that surprised me: even after deciding on opensearch-py, picking which version wasn't obvious, and getting it wrong would have shipped a "fix" that didn't actually fix anything.

Not every opensearch-py release frees urllib3. Early 1.x and 2.x releases inherited the exact same <2 cap we were trying to escape. One release briefly dropped the cap without formally declaring urllib3-2 support, and the very next release re-added it. The real cutover point was version 2.6.0, shipped in PR #719:

We landed on 2.8.0: same migration effort as 2.6.0, but with roughly six more months of bug and security fixes, while still avoiding the breaking change in 3.0 (which makes arguments like body= keyword-only, not removed, just no longer allowed positionally, which matters if any of your call sites pass it positionally).

In dependency terms, >=1.26.19 is a floor: it says "at least this version," not "exactly this version." Package maintainers almost always publish floors rather than pins, because a pin would make their library incompatible with anything else in your project that needs a different exact version. That's exactly why a floor alone can't guarantee what actually gets installed.

The gotcha I'd flag loudest: that urllib3 constraint is a floor, not a pin. 1.26.19 satisfies >=1.26.19 just as well as 2.7.0 does, so installing opensearch-py 2.8 doesn't guarantee your resolver picks a patched urllib3, it just stops forbidding it. Skip a separate urllib3>=2.5.0 pin in your own dependency file, and you can finish this entire migration still shipping the vulnerable version, because the old version still satisfies every constraint in the graph. We pinned it explicitly, with a comment citing the CVE, so the reason survives whoever edits that line next.

# pyproject.toml
[tool.poetry.dependencies]
python = "^3.12"
opensearch-py = "^2.8"

# Explicitly floor urllib3 >= 2.5.0 for CVE-2025-50181 remediation.
# opensearch-py only floors at >=1.26.19, so explicit flooring forces the resolver to upgrade.
urllib3 = ">=2.5.0, <3"

Bridging the DRF Gap: The One Piece of Code We Actually Had to Write

Most of the Python call-site changes are import renames (from opensearchpy import OpenSearch, Q, Search in place of the Elasticsearch equivalents), pure find-and-replace. Django REST Framework integration was the exception: DRF's ordering and pagination classes expect a Django ORM queryset, not an elasticsearch-dsl Search object, so our workflow changelog view relied on django-elasticsearch-dsl-drf to bridge that gap.

That package depends on the old elasticsearch client, reintroducing the urllib3 cap, and hasn't shipped since July 2022, with no OpenSearch successor: django-opensearch-dsl only handles indexing, and django-anysearch-dsl-drf never made it to PyPI.

Rewriting the endpoint onto the ORM would have meant a real backfill, not a client swap, so we vendored the roughly 190 lines we actually used into a project-local module, processor/v1/opensearch_drf.py. If you hit the same wall: check whether a maintained package already covers your usage (indexing-only ones often skip the DRF layer), how narrow your actual usage is, and whether anything downstream depends on an exact response shape.

Preserving the API Contract

The frontend application depended on a rigid JSON response structure for pagination and ordering:

{
  "count": 142,
  "facets": {},
  "next": "https://api.domain.com/v1/.../changelog/events/?page=2",
  "previous": null,
  "results": [...]
}

To maintain 100% backward compatibility without changing frontend integration code, our inline PageNumberPagination implementation handles OpenSearch's response structure explicitly:

# processor/v1/opensearch_drf.py (excerpt)
from rest_framework.pagination import PageNumberPagination as DRFPageNumberPagination
from rest_framework.response import Response
from rest_framework.exceptions import NotFound

class PageNumberPagination(DRFPageNumberPagination):
    """
    Project-local DRF paginator over an opensearchpy Search object.
    Preserves exact contract: {count, facets, next, previous, results}.
    """
    page_size = 50  # Default PAGE_SIZE

    def paginate_queryset(self, queryset, request, view=None):
        page_number = request.query_params.get(self.page_query_param, 1)

        # Calculate search slice
        try:
            page_number = int(page_number)
        except ValueError:
            if page_number == 'last':
                page_number = self._get_last_page_number(queryset)
            else:
                raise NotFound("Invalid page.")

        bottom = (page_number - 1) * self.page_size
        top = bottom + self.page_size

        # Execute sliced opensearchpy Search query
        self.response = queryset[bottom:top].execute()

        # Parse total hits (supports ES7+ / OpenSearch dict/object shapes)
        total = self.response.hits.total
        self.count = total.value if hasattr(total, 'value') else total

        if self.count > 0 and bottom >= self.count:
            raise NotFound("Invalid page.")

        self.request = request
        self.page_number = page_number
        return list(self.response)

    def get_paginated_response(self, data):
        return Response({
            'count': self.count,
            'facets': {},  # Maintained for strict frontend schema contract
            'next': self.get_next_link(),
            'previous': self.get_previous_link(),
            'results': data
        })

By coupling this with an OrderingFilterBackend that transforms user query parameters (?ordering=-timestamp) into OpenSearch .sort({'timestamp': {'order': 'desc'}}) execution calls, we dropped django-elasticsearch-dsl-drf completely and uninstalled four transitive packages.

Two Gotchas Worth Knowing Before You Try This

• An app-name collision broke an import at boot. opensearch-py's optional metrics module does from events import Events, expecting a small PyPI package called Events. Our own top-level Django app was also named events, and it shadowed the real package. The fix was a two-line shim satisfying the import, not a working metrics backend, just enough to stop the crash, since we don't use that feature anyway.
• "No product check" is the point, not a risk to route around. It's the entire reason one client works against both a legacy Elasticsearch-OSS container in CI and a real AWS OpenSearch domain in production. If you ever see UnsupportedProductError after this migration, it means a stray import elasticsearch survived somewhere. A full-tree grep should come back completely clean.

Verification & Observability Strategy

Validating search client migrations requires separating metadata aggregations (size=0) from real document retrieval hits.

In Datadog, we isolated event retrieval traffic using query exclusions:

service:backend-api @logger.name:opensearch -"size=0"

A successful status 200 response on a non-size=0 query confirmed that both ordering backends, pagination slicing, and document mapping round-tripped cleanly through opensearch-py to AWS OpenSearch.

Finally, running pip-audit verified the security milestone:

$ pip-audit
No known vulnerabilities found