UpKeep
UpKeep was already an enterprise product when I joined. It had scaled quickly, and the engineering challenge had shifted from making it scale to making it easier and safer for the team to keep building.The scaling work had left behind architectural and operational debt: client-side data access, inconsistent schemas, expensive releases, slow reporting paths, and increasing coupling as new features were added.I worked hands-on on parts of the application and infrastructure while leading the team toward the larger architectural direction. The work happened across several parallel tracks rather than as a single linear rewrite.
Visit siteHow I contributed
Challenges
- Scaling debt after rapid growth
- The application had already moved from Parse.com to Node.js and MongoDB. Parse's 1,000-result query/page limitation had shaped the product experience — large exports could require users to download multiple CSVs and combine them manually.
- Clients coupled directly to the data layer
- Multiple clients used the client-side Parse SDK to write directly to the database. That allowed schema variations to emerge between clients and raised security concerns about where application logic and data access were enforced.
- GraphQL needed to scale with adoption
- GraphQL was already being adopted as the backend boundary, with some operations going through Parse APIs and others directly to PostgreSQL. As more functionality moved onto it, N+1 request patterns caused performance problems and contributed to frequent downtime. Batching and caching were needed to make the existing GraphQL architecture reliable at growing usage.
- Analytics was a different workload
- MongoDB was useful for the application's operational data, but analytics needed relational queries and aggregation. MoSQL was used as the ETL layer to move the relevant data into PostgreSQL, where reporting workloads could be optimized independently.
- Deployments consumed engineering time
- The application ran on EC2 instances and releases involved deploying to an instance, creating a new machine image, and gradually rolling that image out. A deployment could consume an entire development day.
- The feature surface kept expanding
- Fuzzy search, webhooks, bulk operations, and audit-related capabilities all needed to make use of data already being produced by the application. Building each capability directly into request paths would have increased coupling again and made replaying existing data difficult.
Role & ownership
- LedTechnical direction across application, data, and infrastructure architecture
- LedTeam alignment around the larger architecture while remaining hands-on with implementation
- OwnedScaling and reliability of the existing GraphQL architecture
- OwnedBatching and caching to address GraphQL N+1 request patterns
- OwnedDatabase performance work across PostgreSQL and MongoDB
- OwnedContainerization and migration from EC2-based image deployments to ECS/Fargate
- OwnedEvent-driven architecture built on Apache Kafka
- OwnedFuzzy search implementation using Elasticsearch
- LedTesting environments, infrastructure monitoring, and release reliability
- Contributed toProduct feature work across the Django and React stack
How it worked
- ClientsGraphQL— application requests
- GraphQLApplication APIs— backend logic
- Application APIsKafka— domain events
- KafkaElasticsearch / workers / future consumers— consume or replay
- Application APIsMongoDB / PostgreSQL— data access
GraphQL became the controlled application boundary rather than leaving clients to write directly through the data layer. It could use existing Parse APIs where necessary or access PostgreSQL directly depending on the operation.
Kafka was introduced because new capabilities needed to consume data the application was already producing. Replayability was important: features such as audit logs could be built from existing events instead of requiring every capability to exist when the original transaction happened.
Elasticsearch became a consumer of application data for fuzzy search rather than another transactional source of truth. This allowed search and other consumers to evolve independently from the core request path.
Key decisions
- 01
Make the existing GraphQL architecture reliable before expanding its adoption
- Why
- GraphQL was already becoming the backend boundary, but N+1 request patterns were causing performance problems and downtime.
- Trade-off
- Batching and caching added complexity to the API layer.
- Result
- More features could safely use GraphQL, adoption grew, and the team became less hesitant to ship through it.
- 02
Use MoSQL to feed PostgreSQL for analytics workloads
- Why
- Relational reporting and aggregation were better separated from the operational MongoDB workload.
- Trade-off
- Introduced an ETL path and another datastore that had to remain sufficiently current.
- Result
- Reporting queries could be optimized independently and core query latency moved from roughly 30s to 10–100ms.
- 03
Containerize the application and move from EC2 image rollouts to ECS/Fargate
- Why
- Deployment required an engineer to create and roll out machine images and could consume an entire development day.
- Trade-off
- Required changing the deployment model without a direct user-facing feature payoff.
- Result
- Deployments became faster and more reliable while removing the need to manage EC2 instances; infrastructure cost also fell by around $2.5K/month.
- 04
Introduce Kafka as an event backbone
- Why
- Multiple upcoming features needed the same application data, and replayability would allow new consumers to be built from events already produced by the system.
- Trade-off
- Added operational complexity and moved some capabilities toward eventual consistency.
- Result
- Fuzzy search, webhooks, and bulk operations could be built as consumers without coupling their logic directly to the core request path.
- 05
Use Elasticsearch for fuzzy search
- Why
- Fuzzy search was a product requirement that was better served by a dedicated search engine than by extending the transactional data stores.
- Trade-off
- Added a projection that had to be kept in sync with application data.
- Result
- Fuzzy search could be implemented independently of the core application queries.
Impact
~30s → 10–100ms
Core query latency
~$2.5K/mo
Infrastructure cost saved
1,000
Records per Parse query/page
What changed
- More application functionality moved behind GraphQL as batching and caching made the existing architecture reliable enough to adopt with confidence.
- Schema variations and direct client/database coupling were reduced by moving application logic behind a controlled backend boundary.
- Analytics workloads became practical through MoSQL ETL into PostgreSQL; query latency moved from roughly 30 seconds to 10–100ms and helped enable the KPI Dashboard.
- EC2 image-based releases were replaced with containerized ECS/Fargate deployments, reducing deployment effort and improving reliability.
- Kafka provided a replayable event backbone that enabled fuzzy search, webhooks, and bulk operations to be developed independently of the core request path.
Timeline
- 2018
Joined an already-scaled enterprise product
Inherited a system that had grown quickly and accumulated architectural and operational debt.
- Early engagement
GraphQL adoption and backend boundary
Worked on the existing GraphQL architecture as more functionality moved away from direct client/data-layer access.
- As adoption grew
GraphQL performance and reliability
Addressed N+1 request patterns with batching and caching so more features could use GraphQL reliably.
- In parallel
Analytics workload separation
Used MoSQL as ETL into PostgreSQL and optimized reporting/query paths.
- In parallel
Containerization and deployment migration
Moved from EC2 image-based releases to ECS/Fargate, removing the need to manage the underlying EC2 instances.
- Later
Kafka event-driven architecture
Introduced replayable events so new capabilities could consume existing application data without coupling themselves to synchronous request handling.
- Later
Fuzzy search and other consumers
Implemented Elasticsearch-backed fuzzy search and enabled features such as webhooks and bulk operations through the event architecture.
- 2018–2021
Team leadership alongside implementation
Implemented parts of the system directly while keeping the team's parallel work aligned with the larger architectural direction.
Artifacts
Looking back
The biggest change was not any individual technology. It was moving the system from architecture optimized for rapid growth to architecture optimized for the team continuing to build.
GraphQL established a clearer application boundary, but the work was making an existing adoption scale reliably rather than introducing GraphQL itself.
The infrastructure migration was less visible than product features but removed a recurring engineering cost. Fargate meant the team no longer had to manage the EC2 layer just to deploy the application.
Kafka moved complexity rather than removing it. The trade was deliberate: replayable events and independent consumers made it possible to build new capabilities from data the system was already producing.