[Remote] Senior Engineer - AI
Note: The job is a remote job and is open to candidates in USA. XCUTIVES INC. is looking for a Senior Engineer specializing in AI. This role focuses on building and maintaining data pipelines and infrastructure for AI agent systems, ensuring high-quality data access for AI models and agents, while engaging in client-facing consulting across various industries.
Responsibilities
- Building and maintaining the data pipelines and infrastructure that fuel AI agent systems
- Ensuring that AI models and agents have continuous access to high-quality, timely data
- Handling a wide array of data: financial transactions for BFSI AI solutions, sensor and machine data for Manufacturing AI, patient or research data for Life Sciences AI, and more
Skills
- Strong programming skills, especially in Python, and experience with other languages like SQL
- Practical experience building data pipelines end-to-end
- Proficiency in writing and optimizing SQL queries
- Experience with big data technologies like Apache Spark
- Familiarity with streaming frameworks and tools like Kafka
- Ability to interact with web APIs for data ingestion or extraction
- Strong understanding of data formats and ability to parse JSON, XML, or custom text formats
- Basic DevOps skills, including using Git for version control and CI/CD pipelines
- Strong ability to troubleshoot data issues
- Attentiveness to data quality and skills in implementing checks and validating outputs
- Good communication skills to work with the team and clients
- Ability to handle multiple tasks and prioritize effectively
- Aptitude to learn domain context from data
- Understanding of handling sensitive data securely in pipelines
- Willingness to learn new tools or frameworks as needed
- Experience with tools such as Apache Airflow, Informatica PowerCenter, or cloud-based ones like Azure Data Factory
- Proficiency in Apache Spark and knowledge of Hadoop HDFS
- Strong practical SQL skills and familiarity with relational database systems
- Knowledge of specific systems like MongoDB or Cassandra
- Hands-on usage of Apache Kafka and understanding of consumer group mechanics
- Familiarity with cloud storage services like Amazon S3 and cloud compute for ETL
- Experience building or using connectors to RESTful APIs
- Understanding of various file formats and ability to convert between them
- Comfortable with Linux shell and basic shell scripting
- Using Git for source control and setting up CI pipelines for data projects
- Utilizing monitoring tools for data workflows
- Basics of tools like Excel or Python's Jupyter notebooks for data sanity checks
- Understanding network configurations for data transfer
- Familiarity with PyTest or unittest for writing tests for data transformations
- Experience with tools like JIRA and documentation tools
- Bonus if you understand some AI/ML fundamentals
Company Overview