Developer IV (Big Data Engineer)

Spectraforce Technologies
United States, South Carolina, Columbia
Jun 30, 2025
Job Title: Developer IV (Big Data Engineer) Location: Columbia, SC (Partially onsite - 3 days a week) Duration: 11+ months Job Summary: Duties/About the role: Responsible for analysis, design, programming, and implementation of the most complex applications tasks and projects in the area. Devise feasible, logical procedures to resolve business problems through the use of computer resources. Formulate scope and objectives through research to develop or modify complex systems. Provide technical direction and support in the development and support of business systems software and procedures. Design, code, test and debug the most complex application programs. 20% Provide expertise regarding the integration of applications across the business. Conceive, design, and implement structures and programs to business systems software. 15% Act as an internal consultant, advocate, mentor, and change agent providing expertise and technical guidance on complex projects. Work closely with customers, business analysts, and team members to determine business requirements that drive the analysis and design of quality technical solutions. Ensure solutions are aligned with business and IT strategies and comply with the organization's architectural standards. 15% Provide design recommendations based on long-term IT organization strategy. Make recommendations towards the development of new code or reuse of existing code. Responsibilities may also include participation in component and data architecture design, performance monitoring, product evaluation and buy versus build recommendations. 15% Perform analysis, design, programming, and implementation on systems and procedures to solve complex business or scientific problems. Develop enterprise-level applications and custom integration solutions. Evaluate complex interrelationships in immediate programming area to determine how changes in one program will affect another related area. 15% Develop programming and development standards. Devise new sources of data and develop new approaches and techniques. Involved in the full systems life cycle and is responsible for designing, coding, testing, implementing, maintaining and supporting application software that is delivered on time and within budget. 15% Provide guidance to lower level programmers/analysts. Lead, plan, organize, and/or coordinate complex projects or phases of large projects. Determine and resolve problems with other systems analysts, programmers, and systems users. 5% Test designed programs, verify logic, perform any necessary "debugging," and write the documentation. Day to Day: Review and analyze systems, streamlining processes to ensure efficient, consistent use of all resources and adjusting designs to maximize reusability and reduce costs across subsystems. Provide technology solutions and prototypes to be used as the basis or template for more complicated business/technical requirements. Research, diagnose, and develop fixes for defects in existing code. Address maintenance issues related to current development, testing, and production. Act as an internal consultant, advocate, mentor, and change agent providing expertise and technical guidance on complex projects. Work closely with customers, business analysts, and team members to determine business requirements that drive the analysis and design of quality technical solutions. Ensure solutions are aligned with business and IT strategies and comply with the organization's architectural standards. Provide design recommendations based on long-term IT organization strategy. Make recommendations towards the development of new code or reuse of existing code. Responsibilities may also include participation in component and data architecture design, performance monitoring, product evaluation and buy versus build recommendations. Perform analysis, design, programming, and implementation on systems and procedures to solve complex business or scientific problems. Develop enterprise-level applications and custom integration solutions. Evaluate complex interrelationships in immediate programming area to determine how changes in one program will affect another related area. Develop programming and development standards. Devise new sources of data and develop new approaches and techniques. Involved in the full systems life cycle and is responsible for designing, coding, testing, implementing, maintaining, and supporting application software that is delivered on time and within budget. Provide guidance to lower-level programmers/analysts. Lead, plan, organize, and/or coordinate complex projects or phases of large projects. Determine and resolve problems with other systems analysts, programmers, and systems users. Research application development quality control trends throughout the industry. Work environment: Customer-focused, project-oriented programming environment. Fast paced, multi-platformed environment which may require action and response 24X7 to support the technical business needs of the customer. ? Team Info/Team name: Commercial Systems Information Management (CSIM) aka Data Lake Team Job Requirements: Required Experience: 8 years-of application development, systems testing or other job related experience. Required EDU: Bachelor's degree-in Computer Science, Information Technology or other job related degree. Degree Equivalency: 4 years of job related work experience or 2 years of job related experience plus an associate's degree in Computer Science, Information Technology or other job related degree Required Software and Tools (Hands on experience required): Experience in big data technologies Experience in cloud-based technologies AWS Services - State Machines, CDK, Glue, TypeScript, CloudWatch, Lambda, CloudFormation, S3, Glacier Archival Storage, DataSync, Lake Formation, AppFlow, RDS PostgreSQL, Aurora, Athena, Amazon MSK, Apache Iceberg, Spark, Python Nice to have/Preferred skills: AWS Redshift Databricks- delta Lake Unity catalog Data Engineering and processing using Databricks AI and Machine Learning Amazon Bedrock AWS Sagemaker, AWS Sagemaker Unified Studio R Studio / Posit Workbench, R Shiny /Posit Connect Posit Package Manager AWS Data Firehose Kafka Hive Hue Oozie Sqoop Git/Git Actions IntelliJ Scala