Mark Andreev

Senior Software Engineer

// Contacts mark.andreev@gmail.com Linkedin Github
// Blog Deep dive into Apache Parquet Format How test Java code structure in unit tests (archunit) Java Two Way SSL Client (+ Spring example) [ All snippets, External publications ]
// Talks "ML in production" at the FunTech ML-meetup "Streaming vs Batching" at the Conundrum Meetup
// Projects [App] fix parser [App] .gitignore generator [Demo] Time series player [Demo] Offline lock with Redis Bigdata Indicators Tornado-swagger EnduringNet
// Certificates CockroachDB Query Performance for Developers Redis for Java Developers ScyllaDB. Data Modeling and Application Development AWS Well-Architected Training Deep Dive into AWS S3, Glacer, EFS Deep Dive on Container Security
// Development Stack Java. Spring: MVC, Data, AMQP, Kafka, Integration, Batch, Security, State machine, Apache Camel, Vert.x, GraalVM. Python. Pandas, Scikit-learn, XGBoost, LightGBM, Catboost, Matplotlib, Tornado, FastAPI, Flask Data Processing. Spark, Flink, Cassandra, Hadoop, Kafka, PostgreSQL. Third party. Oracle Database, PostgreSQL, Clickhouse, Kafka, Keycloak, MongoDB, RabbitMQ, Redis, Prometheus, Docker, Kubernetes, Helm, Linux, Airflow Cloud. AWS {EC2, S3, RDS, CloudFront, SQS, SNS, Lambda, Batch, IAM, Registry}; Azure {VM, BLOB, Registry}
// Experience summary

Senior Software Engineer at Man Group (AHL) in London, focused on performance and reliability of trading/OMS infrastructure - from onboarding new asset-class flows to profiling execution paths and tuning Oracle SQL in user-facing trading workflows.

Previously, I spent several years building high-throughput ML/MLOps data platforms (Kafka / ClickHouse feature-store infrastructure, Python SDKs, Kubernetes deployments), after starting my career in machine learning engineering.

​

That ML + data-systems background shapes how I approach trading problems: tight latency budgets, predictable data movement, and pragmatic system trade-offs.

 // Experience Man Group plc, 2024 - now Senior Software Engineer, May 2024 - now
  • Onboarded new asset classes and flows into trading OMS pipeline
  • Tuned Oracle SQL*Plus queries via Autotrace analysis; slashed critical query latency 60%
  • Profiled execution paths with AsyncProfiler; cut user-facing flow latency 70%
  • Adopted Vaadin for internal developer-focused UI | Unified teams off local scripts, accelerating cross-team iteration

[Java, Oracle Database, Kafka, Python, Linux]


Conundrum.ai, 2017 - 2024 Senior Software Engineer, Sep 2022 - May 2024
  • Implemented low-level optimization for Python SDK that helps integration with the platform | Decreased RAM usage by 30%
  • Implemented low-level optimization for feature store on top of Kafka & ClickHouse (projections, application-level query planner) | Reduced execution time of the most popular queries by 63%
  • Covered 80% of queries to eliminate major performance degradations (Gatling)
  • Created performance optimizations for Kafka subscription proxy (Java 21 virtual threads, shared subscription) | Decreased CPU load by 80%
  • Implemented security improvements for Platform (audit, L4 network policies, L7 network filter) | Apply security IS requirements at network level
  • Instrumented platform’s services with health performance metrics (Prometheus, Grafana, Alerts) | Decrease issue investigation time by 64%

[Java, Python, Query optimizations, Load tests, Kafka, Clickhouse, Kubernetes, Feature store, Spring, Vert.x, GraalVM]

Senior Software Engineer, Sep 2019 - Aug 2022
  • Developed microservices with Spring Boot, PostgreSQL and gRPC
  • Migrated feature store to Kafka & ClickHouse (columnar OLAP database) | Decreased query execution time by 90%
  • Created low-level connectors for Industrial Data Exchange formats (MQTT, OPC UA, Historian) | Decreased CPU load to exchange server by 68%
  • Migrated the model-serving runtime to Kubernetes (Kubernetes API, Helm)
  • Deployed platform to AKS (Azure Cloud) & K3s (on-premises, no internet)
  • Migrated to Keycloak (SSO) for Authz

[Java, Python, Kubernetes, Helm, Kafka, Clickhouse, Azure, AKS, OLAP, Feature store, Spring]

Mid-level Software engineer, Nov 2017 - Sep 2019
  • Created feature store for sensors’ time series data (Java, Spring, PostgreSQL, TimescaleDB)
  • Created model serving runtime server (Python, Processes)
  • Created incident management service (Java, Spring, Spring State Machine)
  • Created ETL pipelines using S3, SQS, S3-SFTP

[Java, Spring, Python, AWS, ETL, Feature store, PostgreSQL, TimescaleDB, State machines, S3, SQS, S3 SFTP]

Junior Machine learning engineer, May 2017 - Oct 2017
  • Airline data clustering; created an approach for data splitting for offline A/B tests
  • Telecom data churn; offline churn scoring based on telecom data activity
  • Web data gender detection; offline gender detection based on web activity
  • Mobile data geo-analysis; created reports about geolocation activity based on mobile location data
  • Industrial time-series data; created data pipeline for failure prediction

[Python, PySpark, Spark, SQL, Scikit learn, Pandas, XGBoost]

Big Data Indicators Internship, Oct 2016 - May 2017 Machine Learning Engineer
  • Created data collection & processing pipeline
  • Used topic models to discover trends
  • Created sentiment analysis models for trend prediction

[Data mining, Python, MongoDB, Redis, Machine learning, NLP, Topic models]

// Education Lomonosov Moscow State University Master's degree. Computational Mathematics and Cybernetics Moscow Power Engineering Institute Bachelor's degree. Institute of automatics and computer science
// Contribute to Open Source
  • Apache Spark
    • [SPARK-49044][SQL] ValidateExternalType should return child in error.
    • [SPARK-49490][SQL] Add benchmarks for initCap.
    • [SPARK-49549][SQL] Assign a name to the error conditions _LEGACY_ERROR_TEMP_3055, 3146.
  • Apache Airflow
    • [AIRFLOW-43853] Add logging support for init containers in KubernetesPodOperator (#42498)
    • [AIRFLOW-43847] Add random_name_suffix to SparkKubernetesOperator (#43800) (#43847)
    • [AIRFLOW-43840] Fix logs with leading spaces in the Docker operator (#33692) (#43840)
    • [AIRFLOW-27553] Add ipc_mode for DockerOperator (#27553)
  • Apache Ignite
    • [IGNITE-13713] Implemented target encoding preprocessor.
    • [IGNITE-13714] Implemented catboost inference integration.
    • [IGNITE-13386] Implemented new distances (BrayCurtis, Canberra, JensenShannon and etc).
  • Keycloak
  • Apache Camel
    • [CAMEL-16092] Add fix for header override by Azure Storage Blob download consumer.
  • Clickhouse
  • Vaadin Flow
    • [FLOW-20058] fix: add nodeVersion in gradle plugin settings.
  • Tornado Swagger
    • [REPOSITORY] Swagger API Documentation builder for tornado server.
// Publications A New Approach to Determining the Attitude of Authors of Short Texts to the Topics Discussed in the Texts on the Example of Estimating the Inflations Expectations, Oct 2017 DATA ANALYTICS AND MANAGEMENT IN DATA INTENSIVE DOMAINS, DAMDID / RCDL’2017, Andreev M. Big Data approach to measure inflation expectations: the case of the Russian economy, Jul 16, 2017 IFABS 2017 Oxford Conference, Goloshchapova I., Andreev M. Measuring inflation expectations ofthe Russian population with the help of machine learning. Voprosy Ekonomiki. 2017;(6):71-93. (In Russ.), Goloshchapova I., Andreev M.
// Social Media
leetcode it52 bebee chess web2rise digitalocean substack openhub reddit tenchat hackmd linktr.ee devpost.com