Abstract
The use of machine learning (ML) models to describe material properties has become an important research direction in materials science and relevant and useful datasets are becoming more and more available. However, the systematic treatment of uncertainties associated with measurement errors remains still a challenging topic. This thesis provides an investigation on the impact of measurement errors on the prediction accuracy of ML algorithms for the mechanical properties of steels.
In the first part this topic is addressed using artificially generated linear and non-linear datasets. It is shown that the errors in the target variable y , although affecting prediction accuracy more strongly than the errors in the features X, average out if y is normally distributed. Errors in the features X, on the other hand, lead to bias in relation to the true correlation that systematically offsets the prediction with respect to the true value.
In the second part, this thesis aims to provide a deeper understanding of the nature of the prediction error via the bias-variance decomposition. It analyses how separate determination of the measurement error and investigation of learning curves can be used to quantify bias or variance of the ML model. This is demonstrated on a martensite starting temperature (Ms) data set and a r-value data set. For the Ms dataset, it is found that for underparametrized models such as linear regression, the training error and validation error converge well above the measurement error indicative of bias. For overparametrized models such as XGBoost or Random Forest, the training error is smaller than the measurement error, while the validation error is sizably higher. Therefore, these models exhibit variance and would benefit from more data points. The performance on the validation set of XGBoost, Random Forest, and Gaussian Process regression is comparable. For the r-value dataset, XGBoost reveals behavior similar to that of the Ms dataset. Finally, an analysis of models available from the literature to predict the mechanical properties obtained from tensile testing reveals that the model's validation error is close to the measurement error. Therefore, the bias and variance of these models are very small and, therefore, the prediction accuracy is much higher than the validation error (or the associated coefficient of determination R2) suggests.
In the first part this topic is addressed using artificially generated linear and non-linear datasets. It is shown that the errors in the target variable y , although affecting prediction accuracy more strongly than the errors in the features X, average out if y is normally distributed. Errors in the features X, on the other hand, lead to bias in relation to the true correlation that systematically offsets the prediction with respect to the true value.
In the second part, this thesis aims to provide a deeper understanding of the nature of the prediction error via the bias-variance decomposition. It analyses how separate determination of the measurement error and investigation of learning curves can be used to quantify bias or variance of the ML model. This is demonstrated on a martensite starting temperature (Ms) data set and a r-value data set. For the Ms dataset, it is found that for underparametrized models such as linear regression, the training error and validation error converge well above the measurement error indicative of bias. For overparametrized models such as XGBoost or Random Forest, the training error is smaller than the measurement error, while the validation error is sizably higher. Therefore, these models exhibit variance and would benefit from more data points. The performance on the validation set of XGBoost, Random Forest, and Gaussian Process regression is comparable. For the r-value dataset, XGBoost reveals behavior similar to that of the Ms dataset. Finally, an analysis of models available from the literature to predict the mechanical properties obtained from tensile testing reveals that the model's validation error is close to the measurement error. Therefore, the bias and variance of these models are very small and, therefore, the prediction accuracy is much higher than the validation error (or the associated coefficient of determination R2) suggests.
Translated title of the contribution | Die Auswirkungen von Messfehlern auf das maschinelle Lernen im Bereich der Stahlwerkstoffe |
---|---|
Original language | English |
Qualification | Dipl.-Ing. |
Awarding Institution |
|
Supervisors/Advisors |
|
Award date | 20 Dec 2024 |
DOIs | |
Publication status | Published - 2024 |
Bibliographical note
no embargoKeywords
- machine learning
- measurement errors
- prediction accuracy
- steels
- linear datasets
- non-linear datasets
- target variable
- feature errors
- bias-variance decomposition
- learning curves
- martensite starting temperature
- linear regression
- XGBoost
- Random Forest
- Gaussian Process regression
- overparametrization
- validation error
- training error
- tensile testing
- coefficient of determination
- root mean square error
- RMSE