Hi, I am Arthur

Arthur Lin

Data Engineer at Modovision Co.

I am a passionate data engineer with 2 years of working experience.

Airflow
Python
Docker
Spark
GCP

Projects

Airflow
Airflow
Contributor Dec 2020 - Present

Production-Grade Scheduling and Management.

Star
Automatic Feeder
Automatic Feeder
Developer Jan 2021 - Present

Automation with Raspberry Pi with google drive as well as LINE

Star
Datathon Competition
Datathon Competition
Developer Sep 11 2020 - Sep 13 2020

Interface of connection with Line Chatbot

Star
Toha
Toha
Owner Jun 2019 - Present

A Hugo theme for personal portfolio.

Star

Recent Posts

Card image cap
Docker Installation

What is Docker? Docker is a set of platform as a service products that use OS-level virtualization to deliver software in packages called containers. Containers are isolated from one another and bundle their own software, libraries and configuration files; they can communicate with each other through well-defined channels. What is Docker Engine? Docker Engine is a client-server application with these major components: A server which is a type of long-running program called a daemon process (the dockerd command).

January 9, 2021 Read
Card image cap
Airflow Dags

Airflow Contents Installation Dags What is DAG Directed Acyclic Graph (DAG) Directed - If multiple tasks exist, each must have at least one defined upstream (previous) or downstream (subsequent) tasks, although they could easily have both. Acyclic - No task can create data that goes on to reference itself. This could cause an infinite loop that would be, um, it’d be bad. Don’t do that. Graph - All tasks are laid out in a clear structure with discrete processes occurring at set points and clear relationships made to other tasks.

January 5, 2021 Read
Card image cap
Airflow Installation

What is Airflow ? Airflow is an open-source workflow management platform written in Python Requirements Python3 Any operating system (Linux, Windows, MacOS) Step 1. Preparation Here I will take Ubuntu for an example. First of all, I will highly recommand you to use virtualenv (venv) sudo apt update && \ sudo apt install -y python3-venv Step 2. Create Python Environment Secondly create a folder airflow and create your env

January 4, 2021 Read

Achievements

Best Presenter

Champion

Graduation

Award Winner