[Remote] Senior Data Platform Engineer
Note: The job is a remote job and is open to candidates in USA. Stack AV is developing revolutionary AI and advanced autonomous systems to enhance safety and efficiency in the trucking transportation industry. The Senior Data Platform Engineer will design and operate high scale data systems, enabling engineers to efficiently run compute and data intensive workloads on Stack AV infrastructure.
Responsibilities
- Design and operate distributed storage systems for scheduling and executing large-scale batch workloads
- Build and maintain an open source, modern data platform
- Optimize utilization of storage resourcesImprove reliability and fault tolerance of large-scale storage systems and data platform components
- Collaborate with teams across the company to understand workload requirements and improve platform capabilities
- Contribute to platform tooling, automation, and CI/CD workflows
Skills
- 7+ years of experience building and operating distributed storage systems or modern data platforms
- Experience operating streaming platforms such as Kafka or Pulsar
- Fluent in Python, and SQL, with experience writing and maintaining highly available data applications using Trino and Apache Spark
- Knowledge of table formats (Iceberg, Delta Lake, Hudi, Xtable)
- Experience operating and optimizing at least one RDBMS (Postgres, MySQL)
- Strong debugging and problem-solving skills in complex distributed systems
- Ability to collaborate across teams and communicate technical concepts clearly
Company Overview