- Identify the most appropriate data sources to use for a given purpose and understand their structures and contents, in collaboration with SMEs when required
- Extract structured and unstructured data from the source systems (relational databases, data warehouses, document repositories, file systems, …), prepare such data (cleanse, re-structure, aggregate, …) and load them onto Hadoop.
- Actively support reporting teams in the data exploration and data preparation phases. Where data quality issues are detected, liaise with the data supplier to do root cause analysis
- Contribute to the design, build, and launch activities
- Ensure the maintenance and support of production applications
- Liaise with Technology Services teams to address infrastructure issues and to ensure that the components and software used on the platform are all consistent
- Experience with understanding and creating data flows, with data architecture, with ETL/ELT development and with processing structured and unstructured data
- Proven experience with using data stored in RDBMSs and experience or good understanding of NoSQL databases
- Ability to write performant SQL statements
- Ability to analyze data, to identify issues like gaps and inconsistencies and to do root cause analysis
- Knowledge of Java Understanding of the Hadoop ecosystem including Hadoop file formats like Parquet and ORC
- Experience with open-source technologies used in Big Data analytics like Scala, Spark, Pig, Hive, HBase, Kafka etc.
- Ability to write MapReduce & Spark jobs
- Knowledge of Cloudera
- Experience delivering scripts
- Experience in working with customers to identify and clarify requirements
- Ability to design solutions that are fit for purpose whilst keeping options open for future needs
- Strong verbal and written communication skills, good customer relationship skills
- Have a true agile mindset, capable and willing to take on tasks outside of her/his core competencies to help the team
- Strong technical skills and a strong interest in the financial industry.
Please be aware we are not necessarily expecting each candidate to cover all the above listed technologies.
- Bachelor’s degree