CV

Data engineer working on Australian energy market data.

General Information

Full Name Gayashan Niroshana Amarasinghe
Current Role Data Technical Lead, ZEN Energy
Location Echuca, VIC, Australia
Contact gayashan.amarasinghe [at] gmail [dot] com
Links LinkedIn · GitHub · Google Scholar
Languages English, Sinhala

Skills

  • Energy market domain
    • AEMO NEM market data (MMS), MSATS, and PDR. NEM constraints and constraint binding.
  • Languages and data
    • Python, SQL, PySpark, Java, C++, Bash. Apache Spark, pandas, NoSQL.
  • Platforms and tooling
    • Databricks, Delta Lake, Azure Data Factory, Databricks Workflows.
  • Automation and AI
    • ELT/ETL pipeline automation. AI-assisted development (Claude Code, Databricks Genie), and a habit of tracking the AI and tooling trends worth adopting to speed up delivery.
  • Ways of working
    • Data migration and integration, microservices, RESTful APIs, Agile/Scrum, TDD, and cross-functional collaboration with product and engineering.

Experience

  • May 2023 - Present
    Data Technical Lead
    ZEN Energy, Australia
    • Took on growing ownership of the data platform across three years and three roles, moving from building pipelines to designing and running the platform end to end.
    • Lead a small team of a senior energy analyst and a data engineer, while staying hands-on with the platform myself.
    • Role progression
      • Data Technical Lead (Mar 2026 - Present)
      • Senior Data Engineer (Mar 2025 - Mar 2026)
      • Data Engineer (May 2023 - Mar 2025)
    • Serve as the in-house subject matter expert for AEMO market data (MMS) and for managing robust data delivery and retrieval through the PDR application and the MSATS system, including the reference-data and mapping conventions that keep records consistent.
    • Handled the migration of the enterprise data platform from SQL Server to Databricks, and productionised end-to-end pipelines covering ingestion, transformation, validation, and reporting across forecasting, trading, and regulatory workflows.
    • Redesigned the high-volume 5-minute market and meter data models that downstream processes depend on, improving performance and reliability of the pipelines feeding them.
    • Built pipelines that ingest and reconcile data from external and third-party sources (energy market systems, CRM, billing, weather, and market-curve providers) into a single analytics layer, handling varied and often messy source structures.
    • Re-architected the platform's backup and recovery framework to be modular and configurable, driven by data criticality rather than a fixed schedule.
    • Built AI-assisted development into the day-to-day pipeline workflow (Claude Code and Databricks Genie), cutting the manual effort in building and testing pipelines, and applied that same approach to sharpen testing, code review, and data quality practice.
    • Set the pipeline design, data-modelling, and orchestration patterns the rest of the team builds against, and work directly with other engineers and analysts to get them applying those patterns independently.
    • Selected work
      • Virtual Tolling Agreement settlement system: one of the first settlement systems for a Virtual BESS. Validates trader-submitted nominations, computes state of charge across dispatch intervals, and settles dispatched energy against capacity, cycle-count, and charge/discharge constraints. In production.
      • Enterprise data platform migration from SQL Server to Databricks: core data workloads rebuilt as bronze, silver, and gold layers on Delta Lake, orchestrated through Azure Data Factory and Databricks Workflows.
      • Earnings model and report: replaced a SQL Server process that took close to six hours and was a major reporting blocker. The new solution parallelises the computation across notebooks and runs in about 45 minutes, which makes frequent runs practical.
    • Tech stack
      • Azure Data Factory, Databricks, Delta Lake, PySpark, Python, SQL, Databricks Workflows
  • Jun 2021 - Jan 2023
    Senior Lecturer
    University of Moratuwa, Sri Lanka
    • Taught and supervised across software engineering and distributed systems, mentoring students and junior staff and building a habit of explaining technical concepts to non-specialist audiences.
    • Undergraduate teaching
      • Distributed Systems, Programming Fundamentals, Database Systems, Software Engineering, Software Architecture and Design
    • Postgraduate teaching
      • System and Network Design, Cloud Technologies and Systems, Software Development on Cloud
    • Automated grading
      • Initiated and led the project on enabling automated grading methods to expand teaching support at the university.
      • Directly helped deliver programming modules to large classes (1000+ students) and helped implement the open.uom.lk platform.
    • Acted as an external software and data engineering consultant for multiple public sector projects.
    • Key member of establishing the research collaboration between CERN (The European Organization for Nuclear Research) and the Department of Computer Science and Engineering, University of Moratuwa.
    • Supervised multiple undergraduate and postgraduate projects.
  • Jun 2020 - Dec 2020
    Research Fellow
    RMIT University, Melbourne, Australia
    • Designed the software architecture and completed the initial implementation of BIPV Enabler, a building-integrated photovoltaic modelling tool for the construction industry.
  • Oct 2019 - Dec 2019
    CS Research Officer
    RMIT University, Melbourne, Australia
    • Implemented a blockchain based proof of concept, on the Hyperledger Fabric framework, for managing the process of cladding procurement in a construction project.
  • Jun 2015 - Jun 2021
    Lecturer
    University of Moratuwa, Sri Lanka
    • Undergraduate teaching
      • Programming Fundamentals, Database Systems, Software Engineering
    • Automated grading
      • Initiated and led the project on adopting Virtual Programming Lab as an automated grading method to expand teaching support at the university.
      • Directly helped deliver programming modules to large classes (1000+ students).
    • Acted as an external software and data engineering consultant for multiple public sector projects.
    • Key member of establishing the research collaboration between CERN (The European Organization for Nuclear Research) and the Department of Computer Science and Engineering, University of Moratuwa.
    • Supervised multiple undergraduate projects.
    • Was on paid study leave from 2017 to 2020 to complete my doctoral studies.
  • Mar 2014 - May 2015
    Software Engineer
    WSO2 Inc., Sri Lanka
    • Designed, built, and tested Java components for the WSO2 Carbon Kernel, the core platform behind the WSO2 ESB and API Manager, including a Log4j 2 upgrade and distributed-caching optimisations.
    • Provided deployment and software support for customers, both on-site and off-site.
    • Tech stack
      • Java, OSGi, SOA, Maven, Log4j, Hazelcast
  • Jun 2012 - Dec 2012
    Intern Software Engineer
    KISTERS AG, Aachen, Germany
    • Implemented a MOD4 protocol driver in C++ for a wide-area sensor network monitoring air quality. Part of the IAESTE student exchange program.
    • Tech stack
      • C++

Selected Projects

  • 2026 - now
    NEM Constraints
    • Models AEMO NEM network and FCAS constraints and their binding, evaluating constraint equations against dispatch conditions to determine which constraints bind and when. Live at nem.amrsnghe.org.
  • 2021 - now
    MCQAutoGrader
    • Automated grading for paper-based MCQ examinations, adopted into the grading workflow at the Department of Computer Science and Engineering, University of Moratuwa, and estimated to save around LKR 1M a year in manual effort.
  • 2017 - 2021
    ECSNeT++
    • Open-source simulation toolkit for distributed stream-processing applications on edge and cloud computing environments, produced as part of my doctoral research.
  • 2020 - 2023
    BIPV Enabler
    • Building-integrated photovoltaic modelling software for the construction industry, built at RMIT University.

Education

  • 2017 - 2021
    PhD, Computer Science
    The University of Melbourne, Melbourne, Australia
    • Thesis
      • Distributed Data Stream Processing and Task Placement on Edge-Cloud Infrastructure
    • Summary
      • Designed and developed novel algorithms to optimise the deployment of data stream processing ETL pipelines and low latency applications on cloud and edge computing infrastructure, using constraint satisfaction solvers, integer linear programming, and Markov models.
  • 2009 - 2014
    B.Sc. Eng. (Hons.), Computer Science and Engineering
    University of Moratuwa, Sri Lanka
    • First Class Honours, with a GPA of 3.94/4.2 and a class rank of 3/100.

Certifications

Honors and Awards

  • 2022
    • Excellence Award, awarded first position at the 21st European Conference on e-Learning for the open.uom.lk e-learning and knowledge sharing platform, of which I was a founding member.
  • 2017
    • Elite PhD Scholars' Programme, a joint PhD scholarship scheme awarded by The University of Melbourne and the University Grants Commission of Sri Lanka.
  • 2013
    • Full Scholarship, to participate in the 3rd South Asian Workshop on Research Frontiers in Computing at the School of Computing, National University of Singapore.
    • The Migara Ratnatunga Trust Award, awarded by the Institution of Engineers, Sri Lanka, for excellent performance in the Undergraduate Internship Program.
  • 2011
    • Sri Lankan Finalist, Microsoft Imagine Cup, in the "Software Design" category.
  • 2009 - 2014
    • Dean's List Distinction, awarded for achieving a semester GPA of 3.8 or above (Semesters 1, 2, 3, 4, 7, and 8).
  • 2008
    • The Dr. G. C. L. De Silva Memorial Prize, Royal College, Colombo, awarded for achieving distinctions in all main subjects at the G.C.E. A/L and G.C.E. O/L examinations.

Interests

  • Cycling (road, gravel, and cross-country)
  • Reading (science fiction and biographies)