Teaching Data Analytics from probability to prediction

Getting to teach data analysis for undergraduate and graduate students at the same time is rare. I have that opportunity this semester and am very much looking forward to it. My two courses, Civil Engineering Data Analysis (CE 264) for sophomores, and Advanced Data Analysis (CE 1101) for seniors and higher are designed to understand and model big data often encountered in engineering.

I like how CE 264 was intended to teach the concepts of probability and statistics for Civil Engineers. We start with various types of data that Civil Enginers often encounter; for instance, loadings on beams and columns and stress-strain data for structural engineers, flow data, and water quality data for water resources and environmental engineers, and ridership or traffic data for transportation engineers. We look at how one can understand the data in the context of developing models for civil engineering design. We discuss the standard parametric methods available in the textbook. We also learn non-parametric techniques which are not discussed in the book. Non-parametric methods are especially useful when identifying a probability distribution is difficult and where the sample sizes are small, as with many civil engineering data. The homework problems, lab exercises, and the project are also designed using civil engineering data from real projects. One of the unique features of this class is its computer lab time using RStudio every week. The labs are developed to better understand the concepts from the lectures through actual data and simulations.

In the graduate class, we have in-depth coverage of exploratory data analysis including sampling issues, measurement error, estimation of frequency/probability distributions, resampling, and bootstrap confidence or prediction intervals of a process. We also learn dependence measures, trends in space and time, dimension reduction techniques, and frequency domain models. Finally, we build cross-validated predictive models using linear models and basis functions and model free nonparametric nearest neighbor methods.

I am hopeful that these courses will instill in students, an interest in data analysis.

When, where and how

to make optimal decisions is what my students from CE 316, Civil Engineering Decision and Systems Analysis learned during Fall 2016. They are now proficient in linear, nonlinear, integer, mixed integer and multiobjective programming/optimization. They also know how to solve network problems like shipping goods, shortest paths, and maximizing flow. They can use the same network structures to make sequential decisions under uncertainty and optimally schedule construction jobs and complete them under budget, ahead of schedule. They know the time value of money and can tell you if a project is beneficial or not in the long run. I am happy to present a few snippets of their term projects, which were identified independently and completed successfully with minimum supervision.

Bidding for Projects: Is your company in constant confusion on what projects to bid for? Are you worried that the projects cannot be completed on time? Soon to be engineers from CCNY have a solution for your problem. Based on the planned duration of any project, they can help you select appropriate and optimal number of projects that will maximize your expected profit under various uncertainties.

Domestic and International Procurements: Do you know which are the best source companies that can supply required quality material (construction or otherwise) at the least cost? Do you want to hire a third party to verify the quality of the material? Don’t worry. We have a sequential decision software to help you pick the best company to procure material from and an associated testing company for quality control.

Operating Water System: Whether you are living in New York City, or in the Catskill area, you can relax, sit back and enjoy the best quality water, even during a drought. Our specialists are at work in satisfying all our competing needs.

CCNY is Starving: With limited food options around the campus, have your ever wondered what to eat to stay healthy and get enough calories to complete the homework, all at a low price? You can do it under $10 per day.

Meal Plan: Are you a high school in the city? Do you know if your daily meal plan is the best? We can give you a nutrition optimized meal plan for high school lunches based on federal regulations and food items approved by the New York City Department of Education.

Where is that food coming from: Do you know what places are best for producing various crops under climate, water, economic and market limitations? Whether you are a farmer or a public planner making water, agriculture, energy policy decisions, we can make your life easier by providing this knowledge in an adaptive framework.

Our secret weapon: Two other secret projects are underway for our design competitions. I will reveal the details of these weapons when we win the competition next semester.


Resolution: Data Analysis Made Easy for a Million People

I will create a platform to make data analysis easy for atleast a million people.

I am not an expert in statistics by any means. I have a Civil Engineering degree with water resources and hydroclimatology background. As a necessity, I picked up the statistics and data analysis concepts from my mentors and several experts in the field. I use them regularly, and at a fairly advanced level in my investigations and research. Over time, I have gained confidence and reasonable expertise that enables me to teach data analysis for undergraduate and graduate level students in an engineering school. I have also been successful in my teaching and somewhat popular among students. I feel well trained and equipped for this battle.

I clearly understand that there may not be a million people who are interested in data analysis. If my mission succeeds, I will have created interest in more than a million people. Whether I succeed or not, is up to Time. If I succeed, the world will be a better place with more analytical people. If I fail, atleast I will fail spectacularly.

Over the next few blog posts, I will reveal more details about the platform.