Web Scraping Engineer - Python
Oxylabs
Full-time
On-site
Vilnius/Kaunas, Lithuania
We’re a team of 500+ professionals who develop cutting-edge proxy and web data scraping solutions for thousands of the world’s best known businesses, including Fortune 500 companies.
What’s in store for you:
You’ll be solving complex challenges and maintaining our own infrastructure with 60PB+ monthly data traffic. Here are its scale and maturity in numbers:
- 6PB+ Ceph storage
- 60PB+ monthly data traffic through our systems
- 300k+ service requests/sec processed
- 500k+ Kafka messages/sec streamed
A word from the team:
We run one of the most advanced and largest scraping and parsing products in the world. We serve thousands of requests per second with a very high success rate. Our scrapers and parsers are used by leading e-commerce, market intelligence, and AI industry players making the work challenging and truly global. The team is a blend of different interesting personalities from different walks of life and nationalities. Here you can find people who are experts in gaming, playing guitar, riding bicycles, and other areas. We, as a team, will support you in learning how to build your own scrapers and will share all the tips, tricks and hacks we know to ensure that you are onboard in no time.
Your day-to-day:
- Develop scalable scrapers.
- Define resilient scraping strategies, unblock websites for scrapingImprove observability in the system.
- Develop back-end solutions for scraping & parsing problems of various magnitudes.
- Maintain the current system and develop new features related to scraping & parsing.
Your skills & experience:
- 2 years of experience working with Python.
- Understanding of computer science, including data structures, algorithms, computability and complexity.
- Version Control skills using Git.
- Knowing your way around asynchronous programming would help you a lot.
- Practical knowledge of any code testing tool.
- Knows how to use Dev tools & Network tab;
- Knows how to write an XPath;
- Web development knowledge - HTML/JS/CSS;
- Knowledge of any front-end framework;
- Experience working with Pydantic, FastAPI, SQLAlchemy, BeautifulSoup, lxml, httpx, Playwright;
- Knows how to use an API;
- Has experience working with MySQL, Redis, Docker, Kubernetes, Elasticsearch, Kibana and monitoring tools like Grafana, Prometheus;
- Has experience in building scalable systems.
Nice to have:
Salary:
- Gross salary: 2200 - 5600 EUR/month. Keep in mind that we are open to discussing a different salary based on your skills and experience.
- To support your professional growth and make you feel taken care of, we’ve put together an expansive benefit package. It covers learning, well-being, celebration, and much more — learn all about it here.
Up for the challenge? Let’s talk!
We may use artificial intelligence (AI) tools to support parts of the hiring process, such as reviewing applications, analyzing resumes, or assessing responses. These tools assist our recruitment team but do not replace human judgment. Final hiring decisions are ultimately made by humans. If you would like more information about how your data is processed, please contact us.