Big Data Engineer
What we do:
Novetta focuses on three core areas: Cyber, Entity, and Multi-INT Analytics. Our products are focused on processing and analyzing vast amounts of data in these core areas. Our services are focused on helping our customers move from complexity to clarity.
The Novetta Difference:
Our interdisciplinary expertise enables us to solve our customers most complex problems through a combination of: Smart people, fresh approaches, and focused inspiration; Domain and subject-matter expertise; Agile and flexible solutions; Software that operates at scale and speed; A culture of challenge and continuous learning. At Novetta, we bridge the gap between what our customers think they can do and what they aspire to achieve.
Our culture is shaped by a commitment to our Core Values:
Integrity Customer Mission Success Employee Focus Innovation
Position Description:
Novetta has an exciting opportunity for a self-motivated and accomplished Big Data Engineer to join our team. Our customer uses their legacy data warehouse as the foundation for a variety of applications supporting intelligence analysis, threat tiering, targeting, entity resolution, and big data analytics. While this warehouse and the associated data pipeline adequately support current production applications, our customer is undertaking a major technology refresh initiative in order to support several new operational analytic priorities, and lay the foundation for next-generation analytics. In collaboration with the team, customer principals, and the solutions architect, the ideal candidate will use their knowledge of the Hadoop ecosystem and other distributed computing technologies to improve the data pipeline and data stores -- and exceed the customer's ingest, ETL, search, alerting, and data visualization goals.
In this role you will:
- Provide design, integration, and operational support for distributed computing and storage technologies to ingest and process extremely large data sources using SQL, NoSQL, Hadoop, and Elastic ecosystem tools.
- Assist with the replacement of an existing commercial search system with a combined solution of Hadoop & Elasticsearch. Assist with the development and implementation of the solution using Elasticsearch 2.x and Hortonworks HDP 2.3 to improve data ingest, indexing, and search workflows.
- Provide day to day administrative of Hadoop, to include installation, maintenance, and configuration.
- Perform troubleshooting and issue resolution for the customer, users, and technical leadership.
- Participate as an engineer on an agile development team, writing, testing, and deploying code to implement changes to existing data pipelines or construct new ones.
- Provide ad-hoc data analysis to support end users.
Basic
Qualifications:
- Bachelor's degree with at least 2 years of experience building and operating big data processing pipelines and/or query solutions.
- Proficient understanding of distributed computing principles.
- Demonstrated ability to manage a Hadoop cluster, and associated services.
- Proficiency with Hadoop, MapReduce, and HDFS.
- Experience with big data ETL frameworks and techniques such as Flume, Oozie, or Sqoop.
- Experience with Big data interactive query technologies such as Spark, Impala, Hive, or Presto.
- Experience with the Elastic ecosystem - elasticsearch, kibana, marvel, watcher, and shield.
- Programming experience, ideally in Python or Java.
- Excellent written and oral communication skills.
Desired Skills:
- Experience with loading and processing data on Hortonworks Hadoop clusters.
- Experience tuning Elasticsearch for ingest and query performance.
- Experience securing Elasticsearch and other big data technologies for the intelligence community.
- Experience with messaging/queuing technologies like RabbitMQ or Kafka.
- Experience with version control platforms (git, svn, etc.)
- Prior experience in support of law enforcement or intelligence related big data solution development.
Security Clearance:
- Must currently possess an active US government Top Secret clearance with the ability to obtain and maintain SCI access within a reasonable, customer-mandated time frame.
- Must be willing and able to pass a counterintelligence (CI) polygraph examination
Get a referral bonus for the great people you know! With our amazing referral program, you could be eligible to win outstanding rewards for referring qualified new hires to Novetta.
Novetta is an equal opportunity/affirmative action employer. All qualified applicants will receive consideration for employment without regard to sex, gender identity, sexual orientation, race, color, religion, national origin, disability, protected veteran status, age, or any other characteristic protected by law.
Estimated Salary: $20 to $28 per hour based on qualifications.
|
|
 | |
