Dziękujemy za korzystanie z infoPraca.pl. Data wydruku 20/04/2021.

Senior Java Developer with Scala

Wroclaw, dolnośląskie

Luxoft Poland Sp. z o.o.

Opis stanowiska pracy

PROJECT DESCRIPTION

You’ll be working with various ETL (Extract Transform Load) pipelines based mainly on Apache Spark in AWS for batch and streaming processing, communicating daily with colleagues distributed locally and abroad. The project includes numerous activities like design, technical supervision, and development. You should be comfortable with discussing selected technical solutions with a customer, as well as assessing design risks, flaws, and eliminating issues. We expect you’ll be able to collaborate technical work among Big Data developers, QAs, and DevOps to ensure the appropriate quality of code, continuous integration, and continuous delivery per best practices of software development.

Project pipelines examples:

  • Spark pipeline to cleanse a dataset for further analysis by ML algorithms by other teams. Deployed on AWS EMR using CDK. Scheduled on a daily basis to process ~1 TB of data
  • Workflow to pull pre-processed data from Teradata database, hash sensitive data and store partitioned data on S3 in parquet format. Scheduled on a daily basis on an on-prem server as an Oozie workflow
  • Hourly Spark pipeline to normalize highly dynamic dataset in JSON format to satisfy SQL standards. Deployed on AWS EMR using CDK
  • Daily Spark pipeline to ingest data to Redshift database for BA analysis. Dataset is extremely dynamic (from 1k columns to 7k each day) which won’t fit a classic SQL database by default

RESPONSIBILITIES
  • Gather requirements from users (Data Analytics), design and implement ETL pipelines from scratch
  • Support and migration of the existing workflows from on-premise to AWS
  • Understanding the Data Lake and Data Warehouse concepts
  • Analysis of the business requirements and task estimation
  • Refactoring of the existing application. Defects fixing
  • Design and implementation of ETL pipelines from scratch on distributed systems
  • Support PROD releases. Release documentation
  • Release automation using AWS CDK and CloudFormation Templates

SKILLS

MUST

5+ years' experience in Java/Scala software development 2+ years' experience with AWS

Tech stack

  • Programming languages: Scala, Java
  • Hadoop: Spark, Kafka, YARN, HDFS, Oozie (Spark is a basis — you have to know Scala)
  • AWS: EMR, S3, Redshift, Glue, Athena, CloudFormation, CDK, CloudWatch, Secret Manager
  • non-AWS: Couchbase, ElasticSearch
  • Data formats: Parquet, Avro, JSON

NICE TO HAVE
  • Programming languages: experience with scripting languages - Python, Groovy, JavaScript
  • AWS: Spectrum, StepFunction, Lambda, SNS, SQS
  • non-AWS: Snowflake, NiFi
CAREER AND PROFESSIONAL DEVELOPMENT
  • Competitive salary
  • Luxoft Training Center
  • Access to e-learning Library
  • Individual development plans
  • Long-term cooperation with possibility to gain experience in different projects and technologies
  • A significant number of professional communities (Agile Community, Tech Community, Business Analysis Community, etc.)
Well-being
  • Group Life Insurance
  • Private healthcare insurance – LuxMed. Unlimited access to specialist consultations with Dental Care included
  • Medical costs reimbursement for employees (up to certain amount per quarter)
  • Travel Insurance
  • Benefit Program - Cafeteria and Multisport Card
Work-Life Balance
  • Various communities (sport, recreation activities, trips, board games, cultural and educational activities)
  • Discounts program for employees – BeLux (banking offer, car leasing offer and cafeteria program discount)
  • Convenient locations in a modern offices with game rooms
  • Multinational projects for clients all over the world

Prezentacja firmy

Luxoft is an emerging global leader in high-end software application development.
... 
Luxoft is a global IT outsourcing and software development company with 4000 employees across 17 locations. It was recently recognized as Leading Central and Eastern European Service Provider in the 2010 Global Services 100 list. Luxoft is famous for flexible delivery models, world's best IT talent pool, low attrition rates, best-of-breed processes and highest security and quality standards.

Luxoft is trusted long-term partner for world class leaders such as Avaya, Boeing, Dell, Deutsche Bank, IBM.
  Rozwiń

Informacje dodatkowe

Ostatnia aktualizacja:
03/03/2021
Wymiar etatu:
Pełny etat
Rodzaj umowy:
Na czas nieokreślony
Liczba wakatów:
1
Min. doświadczenie:
Bez doświadczenia
Min. wykształcenie:
Bez wykształcenia
Branża / kategoria:
Praca IT - Konsulting , Praca IT - Programowanie / Analizy , Praca IT - Hardware / Information Systems
do góry