Data Engineer

Innovative Defense Technologies
$114,000 - $184,000 or $135,000 - $231,000
United States, Virginia, Arlington
4401 Wilson Boulevard (Show on map)
Jun 11, 2025
Description Background Information: Innovative Defense Technologies(IDT), provider of cutting-edge cloud-based integration, automated testing and data analysis for complex, mission-critical systems in the US Department of Defense (DOD), is seeking aData Engineerto be based in our Arlington, VA office.Leveraging advanced software automation, model-based systems engineering (MBSE), and AI-driven capabilities, IDT accelerates capability development, enhances operational readiness, and reduces lifecycle costs for the Department of Defense. The company is dedicated to transforming defense systems through innovation, efficiency, and mission-critical results. Overview: The Data Engineer position offers the opportunity to join our growing team of data scientists and analytics experts. The hire will be responsible for designing and implementing various data pipeline architectures across several projects, as well as optimizing data flow and collection for cross functional teams. The candidate must be experienced in the state-of-the-art data management technologies to support differing data requirements and objectives. Individuals must be self-directed and comfortable supporting the data needs of multiple teams, systems, and products. All applicants must be able to obtain/maintain an active U.S. Security Clearance. Responsibilities Include: Design, implement, deploy, and maintain optimal data pipeline and data management architectures. Assemble large, complex data sets that meet functional / non-functional project requirements Build the infrastructure required for optimal extraction, transformation, and loading of data from a wide variety of data sources using SQL, streaming and 'big data' technologies Implement data pipelines to ingest data to the platform, standardize and transform the data Support the development of analytics tools that utilize the data pipeline to provide actionable insights into the data and optimization of objectives. Create data tools for analytics and data scientist team members that assist them in building and optimizing our product into an innovative industry leader. Work with data and analytics experts to strive for greater functionality in our data systems. Design and architect solutions with Big Data technologies (e.g Hadoop, Hive, Spark, Kafka) Design and implement systems that run at scale leveraging containerized deployments Design, build, and scale data pipelines across a variety of source systems and streams (internal, third-party, as well as cloud-based), distributed/elastic environments, and downstream applications and/or self-service solutions Minimum Required Qualifications: Bachelor's Degree in Computer Science, Computer Engineering, Informatics, Information Systems, or another quantitative field Minimum 5 years of experience in a Data Engineer role Required Skills: Experience with big data tools: Hadoop, Spark, etc. Experience with relational SQL and NoSQL databases, including Postgres Experience with AWS cloud or remote services: EC2, EMR, RDS, Redshift Experience with stream-processing systems: Kafka, Storm, Spark-Streaming, etc. Experience with object-oriented/object function scripting languages: Python, Julia, Java, C++, Scala, etc. Experience with data encryption/security features applied to data-in-transit Advanced working SQL knowledge and experience working with relational databases, query authoring (SQL) as well as working familiarity with a variety of databases Experience building and optimizing 'big data' data pipelines, architectures and data sets Experience performing root cause analysis on internal and external data and processes to answer specific questions and identify opportunities for improvement Experience with development, management, and manipulation of large, complex datasets Experience with database & ETL technologies Demonstrated knowledge of data management competencies and implementation Understands the Big Data related problems and requirements to identify the correct technical approach. Core understanding of Big Data principles and architectural patterns Working knowledge of message queuing, stream processing, and highly scalable 'big data' data stores Experience supporting and working with cross-functional teams in a dynamic environment Preferred Skills: Experience with containerized deployment technologies (Kubernetes, Openshift, etc.) Experience with instantiating and configuring Virtual Machines (VMware, VirtualBox, etc.) Experience with Machine Learning algorithms and applications interfacing with data management solutions Competencies: Solid analytical abilities, coupled with a strong sense of ownership, urgency, and drive Attention to detail Initiative, creativity, reliability, teamwork Ability to adapt to and thrive in a fast-paced environment Pay Range*: $114,000 - $184,000 or $135,000 - $231,000 This amount may not be reflective of actual compensation that may be earned as pay is dependent on a candidate's experience, skills, and education. The posted range does not include bonuses, commissions, tips, or other benefits. IDT is often looking to place multiple candidates at various levels. Therefore, more than one pay range has been included, commensurate with experience. EEO Statement: IDT is an Equal Opportunity employer.