Junior Data Engineer

For our customer we are seeking a junior but skilled Python developer to add features to the framework that automates their data engineering processes. This framework will be used by 100+ data engineers across the enterprise. The ideal candidate will have a strong experience with Python development, software engineering best practices, understanding of data engineering concepts and knowledge of Databricks.

The developer will be responsible for creating config file-based boilerplate code that adheres to best practices in code hygiene, including linting and formatting. Additionally, they will package the code and have a good understanding of CI/CD practices using GitHub Actions.

The ideal candidate for this position is a problem solver with a passion for programming, software engineering best practices and a keen interest in data engineering.

This is a junior position, which also means a lower hourly rate.

Responsibilities
* Add features to Python-based framework for data engineering using OOP concepts and design patterns
* Create and optimize config file based boilerplate code using Pydantic for configuration management
* Adhere to code hygiene best practices, including linting, formatting, and documentation
* Package and distribute Python applications and libraries effectively
* Conduct code reviews to ensure code quality and functional requirements are met
* Leverage the Databricks platform to create automation for build, test, and deploy
* Optimize data engineering processes through automation and other best practices
* Troubleshoot and resolve issues related to the software framework
* Assist in setting up CI/CD practices using GitHub Actions