Data Engineering workshops

Discuss scientific data management with Rucio, a framework for organizing, accessing and processing large datasets distributed across heterogeneous storage and network resources. The workshop brings developer and user communities together around federation, policy-driven data placement, adaptive replication and failure recovery, and the extension of tools developed for ATLAS to other research communities. Community presentations and technical discussion connect operational experience with future software development. 26 October at 12:00 to 30 October at 13:00; Europe/Zurich (CET, UTC+01:00). Participation is available in person and remotely through Zoom. Register by 14 October; remote participants receive connection information following registration. On-site breaks, lunches and social events are supported by the hosts.

This practical introduction develops reproducible scientific workflows using Nextflow. Participants learn channels, processes, operators, executors, configuration, parameters and reusable modules, then apply them to an RNA-seq pipeline. The course shows how containerized dependencies and the dataflow programming model support execution on local machines, clusters and cloud infrastructure. Familiarity with Linux or macOS and a command shell is strongly recommended. Online sessions run 09:30–13:30 CET and the listed fee is €380.

Develop reusable R workflows for collecting research data from public websites. Learn HTML structure and selectors, extract relevant text and tables with R packages, and clean, transform and export the results into analysis-ready datasets. Demonstrations, live coding, quizzes and individual or group exercises develop adaptable scripts. The course prioritises official APIs and addresses scraping when no suitable API exists, including respect for terms of service, copyright and server load. 10–11 December 2026, daily 16:00–19:30, Europe/Madrid (CET, UTC+1). Seven live hours plus three hours of independent exercises. English; working knowledge of R required, with R, RStudio and relevant packages installed. Fifteen places. Regular fee EUR 320, or EUR 256 for Ambassador Institutions. Live attendance is required for the certificate. Payment follows acceptance; advertised fees include VAT.

Build practical skills for organizing and querying relational data with SQL and SQLite. The course introduces database tables, keys and relationships before working through table creation, record changes, filtering, missing values, aggregations and joins. Exercises also cover dates, categories, subqueries and common table expressions; time permitting, participants explore window functions and SQL within reproducible R workflows. No previous SQL or R knowledge is required, and participants receive datasets, setup instructions, code and exercise solutions. Sessions on 8, 15, 22 and 29 January 2027, 11:30–15:00 Madrid time (CET). Fourteen live hours plus eight hours of independent exercises. Current fee including VAT: EUR 573, or EUR 458.40 for Ambassador Institutions. Limited to 15 participants. Acceptance is confirmed before payment. Delivered live online in English with practical exercises. Live attendance is required for a certificate.