Skip to content

Feel free to contact me via LinkedIn or Email

KC's Data & Life Notes

arXiv Analytics Platform

Initializing search

Home
Side Projects
Learning in Progress
Blog
Tags
About Me

KC's Data & Life Notes

Home
Home
Side Projects
Side Projects
- Fraud Detection (Data2ML Ops)
  Fraud Detection (Data2ML Ops)
  - Prerequisites
  - dbt
    dbt
    
    Modeling Data
  - Feast
    Feast
    
    Modeling Features
    
    Deploy Feast on Kubernetes
  - MinIO
    MinIO
    
    Deploy MinIO on Kubernetes
  - MLflow
    MLflow
    
    Deploy MLflow on Kubernetes
    
    Track Hyperparameter Optimization with Optuna and MLflow
  - Ray
    Ray
    
    What, Why, When
    
    How It Works?
    
    In the Bigger Picture
    
    Deploy Ray Cluster on Kubernetes Using KubeRay
    
    Integrate Ray Tune with Optuna, Imblearn, MLflow and MinIO
    
    Submit a Ray Tune Job to Your Ray Cluster
  - KServe
    KServe
    
    What, Why, When
    
    How It Works?
    
    In the Bigger Picture
    
    Install Kserve in Serverless Mode
    
    Deploy Your Model on Kubernetes
    
    Move Feature Retrieval to KServe Transformer
    
    Write a Custom KServe Transformer with Feast Integration
    
    Deploy Your Model on Kubernetes with Feast Transformer
  - Others
    Others
    
    What Is Istio?
    
    What Is Knative?
- Retail Lakehouse
  Retail Lakehouse
  - Kafka
    Kafka
    
    Deployment
  - Apache Iceberg
    Apache Iceberg
    
    Spark Quickstart
    Spark Quickstart
    
    Deployment
    
    Getting Started
    
    Integrated Audits Demo
    
    Table Maintenance Spark Procedures
    
    Write-Audit-Publish with Branches in Apache Iceberg
    
    View Support
  - Trino
    Trino
    
    How It Works?
    
    Deployment
    
    Configure OAuth 2.0 Authentication
    
    Fault-tolerant Execution in Trino
    
    Monitoring with Prometheus and Grafana
    
    Configure Catalogs
    
    Faker Connector
    
    Join Optimization
- Unified SQL-based Data Pipelines
  Unified SQL-based Data Pipelines
- arXiv Analytics Platform
  arXiv Analytics Platform
- Trending Content Prediction
  Trending Content Prediction
- LeetCode
  LeetCode
- Restful APIs with Flask
  Restful APIs with Flask
  - Basics
  - Advanced
Learning in Progress
Learning in Progress
- Monthly Recaps
  Monthly Recaps
- Tech Notes
  Tech Notes
  - Spark
    Spark
    
    How Spark Works
    
    Spark Performance Tuning
    
    Learning Spark (2020)
    Learning Spark (2020)
    
    Ch1 & 2 Introduction to Spark
    
    Ch3 Structured APIs
    
    Ch7 Optimizing and Tuning Spark Applications
  - Iceberg
    Iceberg
    
    Best Practices for Optimizing Apache Iceberg Workloads in AWS
    
    The Definitive Guide
    The Definitive Guide
    
    Ch4 Optimizing the Performance of Apache Iceberg
    
    Migrate from Hive to Iceberg
    
    Disaster Recovery with Apache Iceberg
    
    Iceberg Resources
    
    Apache Iceberg in Production: Insights from Netflix in 2023
  - Flink
    Flink
    
    How Flink Works
  - Airflow
    Airflow
    
    How Airflow Works?
  - Polaris
    Polaris
    
    Apache Polaris Policy
    
    Apache Polaris Resources
  - DDIA
    DDIA
    
    Ch8 transactions
  - Hive
    Hive
    
    Hive Architecture
  - SQLMesh
    SQLMesh
    
    Comparisons: SQLMesh vs dbt Core
  - dbt
    
    dbt
  - Streaming Processing
    Streaming Processing
    
    Streaming Processing Window Types
  - RisingWave
    
    RisingWave
  - Treasure Data
    
    Treasure Data
Blog
Blog
- Archive
  Archive
  - 2025
- Categories
  Categories
  - CI/CD
  - Data
  - MkDocs
Tags
About Me
About Me
- Work Experience
  Work Experience
- Public Speakings
  Public Speakings

arXiv Analytics Platform¶

Project for HackMD Data Engineer 2025

Was this page helpful?

Thanks for your feedback!

Thanks for your feedback!

RAG Applications

2020 Dcard Data Engineering Intern

Made with Material for MkDocs