Platform Data Engineer

remote | Full-time | Fully remote

Apply

We are looking for a mid-level Platform Data Engineer for our Houston Freight Forwarding & Logistics client

This is a full time position - all applicants must be able to work without sponsorship.

This is a remote oppportunity.

Description

 

Essential Functions

You'll use your experience to achieve this in several modes of work, with expectations along these dimension are flexible depending on seniority:

  • Design and architecture—Creating and proposing implementations that prioritize simplicity, reliability, and developer experience
  • Teaching and mentorship—Coaching other engineers and other specialties on how to utilize the data platform or approach complex concepts, whether through documentation, pairing, or other mediums
  • Optimization and review—Paying down technical debt, methodically improving performance and cost, providing high-quality feedback on PRs and RFCs to simplify the surface area of our platform
  • Experimentation—Frequently testing new technologies and approaches, while being discerning about what makes it into production by considering total cost of ownership
  • Troubleshooting—Applying your experience to find and address bottlenecks or unpack complex bugs. This includes troubleshooting in advance with good observability practices!

You will be called on to take a strong role in controlling entropy in our data environment: providing tooling and guidance to prevent data sprawl, excessive system complexity, or the accumulation of technical debt in the codebase. Strategic increases in entropy often need to be balanced out, so our team cycles between innovation (new capabilities) and consolidation (tuning platform experience and efficiency).

Preferred Skills

Required:

  • Experience programming in any language in a collaborative engineering environment, Python preferred
  • Familiarity with infrastructure as code (e.g. Terraform, Pulumi)
  • Strong SQL skills (especially tuning for performance and cost on OLAP query engines)
  • Deep experience with at least one OLAP processing engine such as Spark, Snowflake, BigQuery, Clickhouse, etc
  • Experience with public clouds (e.g. AWS, Azure, GCP, or DO)
  • Ability to reason about and develop designs for data systems and pipelines
  • Experience dealing with governance and security for data infrastructure

Good to have:

  • Experience with Docker & k8s
  • Experience with streaming transport (e.g. Kafka, Pub/Sub) and stream processing
  • Expertise with CDC fundamentals and database replication

Education & Experience

  • Quantitative bachelors’ degree preferred but not required
  • Multiple years of experience in a data engineering or data-heavy backend software engineering role

 

Share this opening with friends