Manohar T Polavaram
Senior Data Engineer
Raleigh,NC
Summary
Senior Data Engineer with 10 years of experience in the designing, developing, and implementing enterprise-level applications. Proficient in Java, Spring, Hadoop, Spark, Machine Learning, AWS, Azure.
Senior engineering professional with deep expertise in data architecture, pipeline development, and big data technologies. Proven track record in optimizing data workflows, enhancing system efficiency, and driving business intelligence initiatives. Strong collaborator, adaptable to evolving project demands, with focus on delivering impactful results through teamwork and innovation. Skilled in SQL, Python, Spark, and cloud platforms, with strategic approach to data management and problem-solving.
Overview
13
13
years of professional experience
5
5
years of post-secondary education
3
3
Certifications
Work History
Senior Data Engineer
Northern Trust
Raleigh, NC
01.2016 - Current
- Designed and implemented scalable data pipelines and applications using Java, Spring Boot, Kafka and NoSQL for Centralized business events.
- Developed Microservices using spring boot framework which transforms data from legacy systems(As/400, Db2) to Snowflake Datamesh for reporting layer.
- Developed PySpark application for processing Loans data from mainframe source to DataLake tables and reported the data to Anti Money Laundering (AML) for federal financial crime investigation SAM alerts
- Developed and optimized ETL workflows in Azure Data Factory, improving data ingestion and transformation efficiency.
- Engineered the migration of legacy data infrastructure to Azure Cloud, reducing operational costs.
- Developed Oracle PL/SQL stored procedures for reporting Java APIs.
- Migrated our enterprise data application from on-premises to Azure cloud and re-designed the application for Azure platform
- Developed data pipelines using Azure Data Factory to process source data to Azure databricks delta tables.
- Developed terraform scripts to deploy environments, configuration in Azure platform.
- Led a team of 5 engineers, mentoring and guiding in data engineering and cloud computing initiations.
- Developed POC for AI/ML use case of foreign exchange domain.
Hadoop Engineer
Visa Inc.
Foster City, CA
06.2015 - 12.2015
- Built Java web services using mvc design pattern to calculate payments data reports across the various data warehouses
- Developed oozie scheduler workflows which creates partitioned hive tables running data quality queries for the ingested data
- Used crontab scheduler to schedule cleaning of older logs and temp data stored on all application servers
- Distributed the resources of the Hadoop file system without any locks or failures for the Hadoop jobs
- Benchmarking and stress testing clusters and calculating max ingestion rate
- Increased the ingestion rate from 200 mb writing to 300 mb per hour on production environment
- Test running various jobs in all Hadoop stack components to check response time, health and performance
- Configuring various Hadoop stack components like Hive, Hbase, Oozie, spark etc
- Cluster maintenance as well as creation and removal of nodes using tools like HDP Ambari
Java Engineer
Wipro Technologies
06.2012 - 12.2013
- Full Life Cycle Development in Distributed Environment Using Java and J2EE framework.
- Responsible for developing and modifying the existing service layer based on the business requirements.
- Developed Oracle PL/SQL stored procedures for retrieval.
- Extensively worked on Database Objects Triggers, Views, Cursors and Indexes.
- Performed integration testing of the Business Logic layer and Data Access layer.
- Created Stored Procedures, functions and views to retrieve the data for client reporting in Oracle.
Education
Master of Science - Computer Science
Northwestern Polytechnic University
Fremont, CA 12.2013 - 04.2015
Bachelor of Science - Computer Science
Jawaharlal Nehru Technological University
Tirupathi 08.2008 - 04.2012
Skills
- Experienced in Java Programming
- Proficient in Python
- Scala Programming
- Unix Shell Proficiency
- SQL
- Spring Boot
- AWS
- MySQL
- Oracle
- Snowflake
- Github
- Jenkins
- terraform
- Docker
- Kubernetes
- HDFS
- Kafka
- Spark
- Hive
- Impala
Azure
Certification
Microsoft Certified Azure Data Engineer
Timeline
USAII Certified Artificial Intelligence Engineer
12-2024
AWS Certified Machine Learning - Specialty
06-2024
Microsoft Certified Azure Data Engineer
04-2024
Senior Data Engineer
Northern Trust
01.2016 - Current
Hadoop Engineer
Visa Inc.
06.2015 - 12.2015
Master of Science - Computer Science
Northwestern Polytechnic University
12.2013 - 04.2015
Java Engineer
Wipro Technologies
06.2012 - 12.2013
Bachelor of Science - Computer Science
Jawaharlal Nehru Technological University
08.2008 - 04.2012
Similar Profiles
Pooja ChandakPooja Chandak
Associate Consultant -Hedge Funds at Northern TrustAssociate Consultant -Hedge Funds at Northern Trust
Pooja ChandakPooja Chandak
Associate Consultant -Hedge Funds at Northern TrustAssociate Consultant -Hedge Funds at Northern Trust
Jack LitherlandJack Litherland
Team Lead Accountant at Northern TrustTeam Lead Accountant at Northern Trust