What is Regression?
Traditional Regression: Historically, regression in geotechnics involved statistical methods to model the relationship between a dependent variable (like bearing capacity of a pile) and one or more independent variables (like soil type, soil friction angle). This was typically done using linear or non-linear regression models based on mathematical formulas and assumptions about data distribution.
Machine Learning Regression: With the advent of machine learning (ML), regression has evolved to leverage algorithms that can process vast amounts of data and uncover complex, non-linear relationships. ML regression can adapt to new data, is less dependent on pre-set assumptions, and, last but not least, can handle higher-dimensional data more effectively.
Machine Learning Algorithms for Regression
In the world of machine learning (ML), algorithms are like different tools, each suited for specific tasks. For someone new to ML, understanding these can be intriguing. Let’s look at three common algorithms used for regression, a type of ML that predicts a continuous value, like estimating the bearing capacity of a foundation in geotechnical engineering.
First, imagine you’re asking a few neighbors about the weather to decide if you need an umbrella. The K-Nearest Neighbors (KNN) algorithm works similarly. It predicts an outcome based on what its ‘nearest’ data points (like neighbors) suggest. In geotechnics, it could mean estimating soil moisture by looking at similar nearby soil samples.
Next, think of trying to walk a straight line while staying as close to the edge of a path as possible without stepping off. This is akin to Support Vector Regression (SVR). SVR finds the best line (or hyperplane in higher dimensions) that represents your data but also tries to stay within a defined margin of error. For example, it could be used to predict the load a pile can bear, ensuring the prediction is neither too high nor too low within a certain error threshold.
Lastly, consider a flowchart that guides you through a series of yes/no questions to reach a decision. This is how Decision Trees (DT) work. They split data into branches at each step, leading to a final prediction. In geotechnical engineering, a decision tree might help in determining the type of foundation system needed based on various soil properties.
Each of these algorithms offers a unique way to look at data and make predictions, making them valuable tools in the AI toolkit for geotechnical applications with their ability to capture complex relationships and effectively handle higher-dimensional data.
Forecasting the Capacity of Piles Using Machine Learning
Our research group has been employing ML to estimate the bearing capacity of piles, using soil parameters and pile dimensions. The figure below is from one of our studies that is under review as of January 2024.
Fig. 1. Performance of top 3 ML models and API classified by soil types
In our study, we employed a range of regression-based ML models like k-Nearest Neighbors (kNN), Support Vector Regression (SVR), and Decision Trees among others. These models were trained and tested on a dataset that included variables such as pile diameter, embedded length, soil type, and other relevant features.
The study revealed that certain ML models, notably the XGBoost, Adaboost and SVR, showed promising results, surpassing the performance of the best performing traditional method, API, for the given dataset. The effectiveness of these ML models was measured using statistical metrics such as Mean Absolute Error (MAE) and Mean Absolute Percentage Error (MAPE). Our findings underscore the potential of ML in geotechnical engineering, especially considering the percentage error was reduced almost 30% compared to the traditional methods.
Recognizing the potential of these models, we are actively working on refining them to enhance their accuracy and reliability. This effort is part of our broader initiative to integrate ML more effectively into geotechnical engineering practices. As we progress, our focus is on continually improving these models, ensuring they become more attuned to the complexities and nuances of real-world applications.