dataautomation 2023.09 - 2023.11

Public Parking Lot Big Data Analysis Project (Seoul City)

Data Engineer

Participated in a project to analyze optimal locations utilizing Seoul City's big data (parking lots, commercial districts, population movement). Constructed a data processing ETL pipeline and an API for ML training using Python and AWS services, and visualized the analysis results on a GIS-based map.

담당 역할

  • Data Engineer: Responsible for the technical design of the overall data pipeline and analysis models, and built and operated an AWS-based automated data collection and processing pipeline.

성과

  • Advanced to the finals and won the Excellence Award at the Seoul City Big Data Contest
  • Realized pipeline efficiency by optimizing large-scale data processing processes.

기술 환경

OS: AWS-based Cloud Environment / DB: NoSQL (AWS DynamoDB) / Languages: Python, SQL / Cloud: AWS Lambda, AWS API Gateway, AWS S3, AWS DynamoDB / Tools: QGIS, VSCode, DBeaver

Tech Stack

Python SQL AWS Lambda AWS S3 AWS DynamoDB QGIS