Software Engineer

Daltix
Beyond data

Job details

Apply now

Sign up to apply

Or sign up to refer and earn a reward of €300

Intro

We are looking for talented profiles to help build and maintain the distributed data collection system that is at the heart of our business. You'll be in the front-lines, facing massive (but interesting!) challenges as we try to scrape all retail data available.

We are a data-driven company which collects and processes more than 600GB of raw data (HTML) daily. We leverage big data technologies such as Serverless, Spark on AWS EMR to crunch these volumes of data and make it queryable.

In this role, you will ensure that our data collection engine, which consists of distributed web crawlers, is state of the art and ahead of our competition. You will ensure that we can scrape any webshop, no matter the ban-detection that has been put in place. Then, it will be important that proper monitoring tools are in place. We are currently scraping 60 sites and your goal is to at least triple that without losing completeness and quality. 

Responsibilities

  • Developing & maintaining web crawlers using Python & JavaScript.
  • Designing & developing internal tools and frameworks.
  • Designing & developing tools to automate testing & QA.
  • Building & maintaining distributed data collection systems on AWS.
  • Guarantee data integrity and quality by extending our logging, monitoring and outlier detection systems.ng

Main requirements

  • Knowledge of Computer Engineering (through formal education or other means)
  • Strong programming experience with one or more languages: Python (preferred), Javascript, C#, Java, Scala, C/C++
  • Good understanding REST API's, databases and SQL
  • Linux command-line experience
  • Highly proficient in spoken and written English
  • You are passionate about software engineering

Nice to have

  • Strong experience with both Python & JavaScript
  • Experience working on top of Amazon Web Services
  • Excellent coding skills and deep knowledge of web technologies
  • Excellent problem-solving capabilities and a critical mindset
  • You get energy from working in a highly complex and challenging startup environment with a high tech product
  • DevOps & automation (Terraform, Ansible)
  • Data analysis using Pandas (Python)

Perks

  • Work with the latest tech stack
  • We're a quickly growing company => you're personal growth can be huge too.
  • Health Benefits – Comprehensive coverage for medical
  • Meal allowance – Monthly meal card along with a fully stocked kitchen with enough coffee and fruits, along with monthly team drinks and dinners
  • Work-Life Balance – We trust you to know your schedule and work when you feel most productive
  • Learning and Development – Attend meet-up, conferences, and events that interest you and benefit your personal and career growth.

Apply now

Sign up to apply

Or sign up to refer and earn a reward of €300