Sr. Data Engineer - Remote (Req. #732)

Founded in 1994, Mindex is a software development company with a rich history of demonstrated software and product development success. Our three divisions – Cloud, SchoolTool, and Software Development – are all rapidly growing, and our employee base is close to 400. We are ranked the #1 Software Developer in the 2023 RBJ’s Book of Lists, the Best Software Developer in the RBJ’s 2022 Reader Rankings, and a 2022 Certified Great Place to Work.

Mindex’s SchoolTool division is responsible for New York State’s market leading Student Management System (SMS), built from the ground up. Our team is involved in the full lifecycle of this product, including sales, training, support, and maintenance.

Mindex is seeking an experienced Senior Data Engineer to join the SchoolTool team. The ideal candidate will be highly experienced in working directly with our product team and customers to design and build interactive dashboards, their underlying data structures (data models and warehouse), and the ETLs (or pipelines) required to populate them.

Essential Functions

• Builds and maintains both technical and non-technical relationships within the team, across Mindex delivery groups, and with customer stakeholders to ensure cohesive collaboration and project success. 

• Evaluates and recommends the most suitable technology stack for each project, effectively communicating trade-offs and guiding internal teams and external stakeholders toward optimal solutions. 

• Applies software engineering principles in the development of analytics solutions using programming languages, databases, and data visualization tools. 

• Architects, builds, tests, scales, and maintains cloud-native data pipelines and data platforms, ensuring performance, security, and reliability. 

• Leads the end-to-end design, development, and delivery of large-scale data solutions, including data discovery, ingestion, transformation, modeling, validation, and deployment. 

• Guides and mentors team members of varying skill levels, fostering continuous learning and promoting engineering excellence. 

• Implements and enforces data quality, governance, and security best practices across all data assets and workflows. 

• Collaborates with data analysts, and business stakeholders to translate analytical and business requirements into scalable technical solutions. 

• Evaluates and optimizes data workflows, architectures, and systems for cost efficiency, performance, and maintainability. 

• Stays current with emerging technologies and industry trends, proactively recommending innovations that enhance data engineering capabilities. 

• Highly Skilled in Python and SQL.

• Strong proficiency in AWS data and analytics services, including AWS Glue, Lambda, S3, Athena, Kinesis, and Redshift. 

• Experience designing and implementing large-scale data warehouses using Snowflake, Redshift, S3 or other data warehousing technologies.

• Expertise in implementing data governance, security, and compliance best practices within cloud environments. 

• Proficiency with OLTP and OLAP database systems, and familiarity with NoSQL technologies such as DynamoDB or MongoDB. 

• Proven experience building and maintaining end-to-end data pipelines for enterprise-scale applications. 

• Experience a plus with Apache Airflow or other workflow orchestration and automation frameworks.

• Experience a plus with distributed data processing frameworks such as Apache Spark and Hadoop. 

• Experience a plus with a strong understanding of DevOps practices and tools such as AWS CodePipeline, GitLab, or Azure DevOps. 

• Experience a plus with Infrastructure as Code (IaC) using technologies such as AWS CDK, AWS CloudFormation, and Terraform. 

• Experience a plus with data visualization tools including Amazon QuickSight, Tableau, and Power BI.