However, rather than using the default hyperparameters assigned by the scikit-lean library, to achieve the primary objectives of the study, hyperparameter optimization was implemented. Hyperparameter optimization is a commonly used practice to build robust ML models. 56,57 The hyperparameters of RFR were tuned using the grid search optimization (GSO) approach. For this, the GridSearchCV optimization algorithm in the scikit-learn library was used. GSO was considered since it runs through all the possible combinations in the hyperparameter space, thus selecting the best combination of the space. 57,58 The hyperparameter space was predefined by including the possible values and it was fed into the GSO algorithm.
GSO was implemented along with random subsampling cross-validation. An approach known as the K-fold cross-validation was used. During the K-fold cross-validation, the training dataset is divided into K number of same-sized portions (folds), and K − 1 of the portions are used for training and the remainder are used for validation. 59,60 This is repeated until each fold gets the chance to be the validation set. For this study, a 5-fold cross-validation was implemented as shown in Fig. 5 . Therefore, the training set was divided into five portions and during each split, four portions were used for training and one portion was used for validation.
Demonstration of the K-fold cross-validation. |
Tuning was done under 4 approaches as shown in Fig. 6 to investigate the effects of the considered hyperparameters. In the first approach, n_estimators was changed from 1 to 10, 1 to 100, and 1 to 1000 in different intervals. The notation used to demonstrate the n_estimators change is shown in Table 2 .
Workflow of the methodology. |
n_estimators change notation | Starting value | Ending value | Increment |
---|---|---|---|
1 | 1 | 10 | 1 |
1 | 1 | 100 | 1 |
1 | 1 | 100 | 10 |
1 | 1 | 1000 | 1 |
1 | 1 | 1000 | 10 |
1 | 1 | 1000 | 100 |
In the second approach, n_estimators was changed from 1 to 1000 in the same way as approach 1 along with max_features . Here, max_features was changed from 10% to 100% of total features in increments of 10%. In the third approach, n_estimators was changed in the same way along with min_samples_leaf . In this case, min_samples_leaf was changed from 1 to 20 in intervals of 1. In the fourth approach, all 3 hyperparameters, i.e. , n_estimators , max_features and min_samples_leaf were varied at the same time in the above-mentioned intervals. In each approach, values of all the other hyperparameters of RFR were kept at their default values assigned by the scikit-learn library. The link to the GitHub folder with the developed codes is given in the appendix.
(7) |
(8) |
In eqn (7) and (8) , y i is the actual value, ŷ is the predicted value, ȳ is the mean value of the distribution, n is the number of data points and m is the number of input features.
(9) |
Model no. | n_estimators change | n_estimators | Adj. R | Computational time (s) | ||
---|---|---|---|---|---|---|
Training | Validation | Testing | ||||
M11 | 1 | 8 | 0.9650 | 0.8188 | 0.8024 | 0.81 |
M12 | 1 | 51 | 0.9760 | 0.8367 | 0.8202 | 70.25 |
M13 | 1 | 51 | 0.9760 | 0.8367 | 0.8202 | 6.88 |
M14 | 1 | 51 | 0.9760 | 0.8367 | 0.8202 | 6932.55 |
M15 | 1 | 51 | 0.9760 | 0.8367 | 0.8202 | 707.56 |
M16 | 1 | 801 | 0.9799 | 0.8352 | 0.8218 | 65.73 |
Adjusted coefficient of determination values of each approach for different changes in n_estimators. |
Interestingly, when the upper limit of the n_estimators range was pushed beyond 100, the performance of the model did not show any noticeable increase in all training validation and testing adj. R 2 values. When n_estimators changed from 1 to 100 in intervals 1 and 10 (models M12 and M13) and n_estimators changed from 1 to 1000 in intervals 1 and 10 (models M14 and M15), the models showed the same performance, i.e. a training score of 0.9760, validation score of 0.8367 and a testing score of 0.8202. However, when the n_estimators changed from 1 to 1000 in intervals of 100, the training and testing scores of the M16 model showed a slight increase in performance, yielding an adj. R 2 of 0.9799 and 0.8218, respectively. However, the validation score showed a slight decrease, which was negligible.
The highest computational time of 6932.55 seconds was shown by the model M14 where n_estimators changed from 1 to 1000 in increments of 1. The results from approach 1 showed that after a certain n_estimators value, the models’ performances increased drastically and the performance was maintained at a constant value over a certain n_estimators range showing that the performance of the RFR when n_estimators was tuned was efficient within a certain range. Since the range and interval at which the n_estimators values are tuned affect the computational time, an effective range and an interval for n_estimators should be decided upon, taking computational time into account.
In approach 2, max_features were also tuned along with n_estimators . Results obtained using approach 2 of the methodology are tabulated in Table 4 . As observed in approach 1, clear spikes in training, validation, and testing adj. R 2 values were observed when the upper limit of n_estimators was increased from 10 to 100. The training score had an increase of 1.36%, the validation score had an increase of 1.92%, and the test score had an increase of 1.60%. This clear jump in performance is noticeable in Fig. 7 . Interestingly, the performances of the models developed in approach 2 were significantly higher than the performance of the corresponding “ n_estimators change” in approach 1. This is quite visible in Fig. 8 . Further, going from approach 1 to 2, the average validation score increased by 2.24% and the testing score increased by 3.52%, which was significant. This increase in adj. R 2 values is an indication that tuning max_features has a major impact on predicting the porosity using RFR. Model M21, where n_estimators were changed from 1 to 10 in intervals of 1 and max_features were changed from 0.1 to 1 in intervals of 0.1, showed the least performance with a training score of 0.9672, validation score of 0.8381, and a testing score of 0.8366. On the other hand, model M23 showed the highest testing performance with an adj. R 2 of 0.8505 where n_estimators changed from 1 to 100 in intervals of 10 and max_features changed from 0.1 to 1 in intervals of 0.1. The model M23 yielded its best test model when n_estimators was 81 and max_features were 0.5. It should be noted that even though model M23 had the highest testing score, the training, and validation scores were not the best out of all the models developed in approach 2. The highest training score of 0.9823 was shown by models M24, M25, and M26. The highest validation scores were shown by models M24 and M25. However, it is more meaningful to select model M23 as the best-performing model since the testing set represents an independent dataset that had never been seen by the model before.
Model no. | n_estimators change | n_estimators | max_features | Adj. R | Computational time (s) | ||
---|---|---|---|---|---|---|---|
Training | Validation | Testing | |||||
M21 | 1 | 9 | 0.1 | 0.9672 | 0.8381 | 0.8366 | 3.69 |
M22 | 1 | 79 | 0.5 | 0.9804 | 0.8542 | 0.8500 | 326.56 |
M23 | 1 | 81 | 0.5 | 0.9806 | 0.8541 | 0.8505 | 30.20 |
M24 | 1 | 520 | 0.5 | 0.9823 | 0.8556 | 0.8467 | 32 |
M25 | 1 | 521 | 0.5 | 0.9823 | 0.8556 | 0.8467 | 3045.27 |
M26 | 1 | 801 | 0.5 | 0.9823 | 0.8554 | 0.8471 | 284.29 |
Adjusted coefficient of determination values for each change in n_estimators for different approaches. |
The anomaly in the validation score observed when the n_estimators were changed from 1 to 1000 in intervals of 100 in approach 1 was also observable in approach 2. The difference in train–test scores provides an idea about the generalizability of the model. The smaller the train–test difference, the higher the generalizability of the model. Overall, the train–test difference in approach 2 was noticeably less than that of approach 1. The average train–test difference decreased by 15.51% on going from approach 1 to 2. This showed that the generalizability of the models improved when max_features was introduced into the hyperparameter space. Similar to approach 1, the highest runtime was shown when the n_estimators changed from 1 to 1000 in increments of 1.
In approach 3, n_estimators was investigated with the alteration of min_samples_leaf , and the results obtained are tabulated in Table 5 . Notably, all the performance results obtained for all the RFR models except the runtimes were the same as that of approach 1, as seen in Fig. 7 and 8 . This was because the optimum value selected by the grid search optimization of the min_samples_leaf was the same as the default value assigned by the scikit-learn library for the RFR algorithm, hence the best testing adj. R 2 was shown by model M34 when the n_estimators was changed from 1 to 1000 in intervals of 100. Computational times were longer than those obtained in approach 1 since models developed in approach 3 had a larger hyperparameter space as compared to approach 1.
Model no. | n_estimators change | n_estimators | min_samples_leaf | Adj. R | Computational time (s) | ||
---|---|---|---|---|---|---|---|
Training | Validation | Testing | |||||
M31 | 1 | 8 | 1 | 0.9650 | 0.8188 | 0.8024 | 7.79 |
M32 | 1 | 51 | 1 | 0.9760 | 0.8367 | 0.8202 | 674.81 |
M33 | 1 | 51 | 1 | 0.9760 | 0.8367 | 0.8202 | 64.96 |
M34 | 1 | 51 | 1 | 0.9760 | 0.8367 | 0.8202 | 70 |
M35 | 1 | 51 | 1 | 0.9760 | 0.8367 | 0.8202 | 6525.18 |
M36 | 1 | 801 | 1 | 0.9799 | 0.8352 | 0.8218 | 606.28 |
Model no. | n_estimators change | n_estimators | max_features | min_samples_leaf | Adj. R | Computational time (s) | ||
---|---|---|---|---|---|---|---|---|
Training | Validation | Testing | ||||||
M41 | 1 | 9 | 0.1 | 1 | 0.9672 | 0.8381 | 0.8366 | 56.22 |
M42 | 1 | 79 | 0.5 | 1 | 0.9804 | 0.8542 | 0.8500 | 4242.86 |
M43 | 1 | 81 | 0.5 | 1 | 0.9806 | 0.8541 | 0.8505 | 425.65 |
M44 | 1 | 520 | 0.5 | 1 | 0.9823 | 0.8556 | 0.8467 | 82 |
M45 | 1 | 521 | 0.5 | 1 | 0.9823 | 0.8556 | 0.8467 | 51 |
M46 | 1 | 801 | 0.5 | 1 | 0.9823 | 0.8554 | 0.8471 | 3796.99 |
Table 7 shows the RMSE values of approaches 1, 2, 3, and 4. While the adj. R 2 values give an idea about the correlation between the actual porosities and the predicted porosities, the RMSE values provide an idea about the difference (or the error) between the two. Therefore, RMSE is also an important parameter in ML model performance evaluation. The pattern in which RMSE values fluctuated in the 4 approaches was similar to that of adj. R 2 . The smallest RMSEs were shown by model M16 with a training model RMSE of 0.9988 and a testing model RMSE of 2.8312. The improvement in the results when max_features was introduced into the hyperparameter space was also evident based on the RMSE values obtained in approach 2. There was a clear decrease in RMSE values in both training and testing models in approaches 2 and 4 where max_features was tuned.
RMSE | |||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|
Approach 1 | Approach 2 | Approach 3 | Approach 4 | ||||||||
Model no. | Training | Testing | Model no. | Training | Testing | Model no. | Training | Testing | Model no. | Training | Testing |
M11 | 1.2894 | 2.9967 | M21 | 1.2516 | 2.7218 | M31 | 1.2894 | 2.9967 | M41 | 1.2516 | 2.7218 |
M12 | 1.0817 | 2.8499 | M22 | 0.9835 | 2.5917 | M32 | 1.0817 | 2.8499 | M42 | 0.9835 | 2.5917 |
M13 | 1.0817 | 2.8499 | M23 | 0.9798 | 2.5875 | M33 | 1.0817 | 2.8499 | M43 | 0.9798 | 2.5880 |
M14 | 1.0817 | 2.8499 | M24 | 0.9399 | 2.6190 | M34 | 1.0817 | 2.8499 | M44 | 0.9399 | 2.6190 |
M15 | 1.0817 | 2.8499 | M25 | 0.9396 | 2.6187 | M35 | 1.0817 | 2.8499 | M45 | 0.9396 | 2.6187 |
M16 | 0.9988 | 2.8312 | M26 | 0.9396 | 2.6148 | M36 | 0.9988 | 2.8312 | M46 | 0.9396 | 2.6148 |
Runtime and grid search combinations had a positive relationship, i.e. , when the number of combinations in the grid search space was the largest, the runtime of the model was the highest, and vice versa . Further, it was observed that from approach 1 to approach 3, the increase in computational times was roughly proportional to each other as seen in Fig. 9 . However, in approach 4 where n_estimators was changed along with the tuning of max_features and min_samples_leaf , an anomaly was observed when n_estimators was changed from 1 to 1000 in intervals of 10.
Runtimes of the models of each n_estimators’ change for different approaches. |
Even though the primary objective of the study was to investigate the influences of n_estimators along with max_features and min_samples_leaf on the performance of RFR, having an overall picture of the variation of the actual and predicted porosity and their relationship is important to understand the model's applicability in porosity prediction. To achieve this, depth-porosity graphs and correlation plots were plotted. Fig. 10 shows one such depth-porosity graph and a correlation plot developed for the best-performing RFR test model (model M23) of the study. The depth-porosity plot indicated that most of the time, the predicted porosity followed the pattern of the actual porosity. The correlation plot showed that the majority of the points were scattered around the perfect correlation line, which is an indication of a high correlation between the actual values and the predicted values.
Depth-porosity and correlation plots obtained from the predictions of the best-performing RFR testing model. |
• Overall, based on both the performance and computational time, the RFR model with n_estimators at 81 and max_features at 2 (while keeping all the other hyperparameters at their default values), which was developed in approach 2, produced the most effective model for predicting the porosity of the Volve oil field in the North Sea with a testing model adj. R 2 of 0.8505, a testing model RMSE of 2.5875, and a computational time of 30.2 seconds.
• There was a notable increase in performance when the upper limit of the n_estimators increased from 10 to 100. On the other hand, the performance of the models did not increase significantly when the upper limit of n_estimators increased from 100 to 1000. This phenomenon indicated that identifying an effective n_estimators range that is not too low (which will make the performance significantly low) and not too high (which will increase the computational time) is important to produce an efficient RFR model during porosity prediction.
• A range of 1 to 100 changed in intervals of 10 can be suggested for n_estimators when developing an RFR model to predict the porosity of the Volve oil field since these models showed higher performances and lower computational times in all four approaches. When the n_estimators range of 1 to 100 was changed in intervals of 10, it always yielded a high adj. R 2 value (in approaches 2 and 4, it yielded the highest testing model adj. R 2 value) for the model and had the second least computational time.
• When n_estimators was tuned along with max_features in approach 2, the results improved drastically as compared to approach 1 where only n_estimators was tuned. There was an average validation score increase of 2.24% and a testing score increase of 3.52% on going from approach 1 to 2. This improvement of the scores (adj. R 2 ) showed that max_features has a significant influence on the RFR model's performance.
• It was observed that computational time was largely affected by the number of hyperparameters altered, their range, and interval. Of all the approaches, the longest computational time was when n_estimators was tuned from 1 to 1000 in intervals of 1 along with max_features and min_samples_leaf .
Based on the results, only by adjusting n_estimators and max_features can an RFR model be developed with a robust prediction power to estimate the porosity in the Volve oil field.
Abbreviation.
AI | Artificial intelligence |
ML | Machine learning |
RFR | Random forest regression |
ANN | Artificial neural network |
SVR | Support vector regression |
LASSO | Least absolute shrinkage and selection operator |
BMA | Bayesian model averaging |
GSO | Grid search optimization |
RMSE | Root mean squared error |
R | Coefficient of determination |
adj. R | Adjusted coefficient of determination |
RES | Resistivity log |
NPHI | Neutron porosity log |
GR | Gamma ray log |
PHIF | Total porosity |
PHID | Porosity from density log |
n_estimators | Number of trees in the forest |
max_features | Number of features considered for the best split |
min_samples_leaf | Minimum number of samples required to be at a leaf node |
max_depth | Maximum depth of the tree |
max_samples | Fraction of the original dataset assigned to any individual tree |
min_samples_split | Minimum number of samples required to split an internal node |
max_leaf_nodes | Maximum leaf nodes to restrict the growth of the tree |
A | A regression coefficient |
B | A regression coefficient |
ρ | Matrix density |
ρ | Measured bulk density |
ρ | Pore fluid density |
n | Number of datapoints |
m | Number of input features |
X | Independent and identically distributed random vector |
θ | Independent and identically distributed random vector |
x | Observed input vector associated with vector X |
Y | A vector with numerical outcomes |
y | Actual value |
ŷ | Predicted value |
ȳ | Mean value of the distribution |
Data availability, conflicts of interest, acknowledgements.
IMAGES
VIDEO
COMMENTS
Specifically, for Iskandar region, the key player of Iskandar Malaysia's economic development is the Iskandar Region Development Authority (IRDA). IRDA was formed under the IRDA Act 2007, Act 664 and it is technically a federal agency which come under the purview of Prime Minister's Department prior to 2018.
Iskandar Malaysia is located in the State of Johor and made up of five local authorities. The region is a leader in sustainable urban development, with a robust climate action "blueprint" for which it has secured backing from local, state, and international governing levels. The Iskandar Regional Development Authority
Case study Case study: A conglomerate acquiring shares in a master developer in ... development plan called "Iskandar Malaysia" was initiated in 2010 by the Malaysian Government as one of the platforms to feature the country's Economic Transformation Programme (ETP). ETP was formulated as part of Malaysia's National
From a regional perspective, the development of Iskandar will lend a greater competitive edge to the region and will benefit significantly from the air and sea linkages within Asia-Pacific countries. Iskandar Malaysia covers an area of about 2216.3 km 2 which is about three (3) times the size of Singapore and two times the size of Hong Kong Island.
Abstract. This paper discusses the process of economic growth and urbanization in Malaysia, the contribution of Johor in the nation's growth and the development of Iskandar Malaysia. First, we ...
The Urbanization and Growth of Malaysia: Case Study of Iskandar Region. M. Y. Yasin, J. Abdullah, +1 author N. M. Noor. Published4 March 2021. Geography, Economics. This paper discusses the process of economic growth and urbanization in Malaysia, the contribution of Johor in the nation's growth and the development of Iskandar Malaysia.
A Case Study of Iskandar Malaysia, Johor ShArIfAh MArIAM MohAMMeD ALhABShI Deputy Vice Chancellor International Institute of Public Policy and Management University of Malaya Kuala Lumpur Malaysia Introduction A "plan" which conveys no targets for reducing poverty, unemployment, and inequality can hardly be considered a "development plan ...
The South Johor Economic Region (SJER), which commonly known as the Iskandar Malaysia (IM), is one of these economic growth centres to be developed as an integrated global node of Singapore and Indonesia (cf. Fig. 10.3 ). Fig. 10.3. Iskandar Malaysia and the surrounding region - Singapore and Riau of Indonesia.
In the early 2000s in the southern region, the Iskandar Malaysia Development Corridor had focused on the low carbon city and society framework (Ho et al., 2013) and it was declared as Iskandar ...
2 emission. This study also prepares a quantitative scenario study on the establishment of low carbon urban development in Iskandar Malaysia. View metadata, citation and similar papers at core.ac.uk brought to you by CORE provided by Universiti Teknologi Malaysia Institutional Repository
This paper aims to examine local and regional resilient policy actions to reduce greenhouse gases using the empirical case of Iskandar Malaysia. The study case is selected because it is one of the ...
Abstract: This paper discusses the process of economic growth and urbanization in Malaysia, the contribution of Johor in the nation's growth and the development of Iskandar Malaysia. First, we explore the range of institutions that engage in urban and regional planning at various level and their respective statutory development plan.
The study used an explanatory sequential mixed approach comprising 4 case studies of conventional buildings, 72 building stakeholders submitted a survey questionnaire, and 7 experts were interviewed to enable triangulation of information. ... The case of Iskandar Malaysia development corridor" by Chin Siong Ho (Ho et al., 2013). Also, In the ...
A low carbon city development program is one of the critical events in today's sustainability initiatives worldwide, including Malaysia. This paper is therefore aimed to review Majlis Perbandaran Klang, MPK framework and their action on the low carbon city development in Klang city, Malaysia based on their recent Diamond Rating Certificate 2017 award by Sustainable Energy Development Activity ...
Eco village concept for green economic development: Iskandar Malaysia as a case study Abstract: Renewable Energy (RE) based on town concept in Iskandar Malaysia (IM) that can serve as a global model for a smart eco — village in tropical countries is proposed. In this research, renewable energy (RE) based distributed energy generation (DEG ...
Erdogan et al., 2007; Lu, 1999). There have been several studies on housing satisfaction in Malaysia, and these studies are focused primarily on conventional homes. However, to date, less empirical studies has been conducted to appraise the performance of the quality of green homes in the country.
PDF | On Oct 20, 2022, Marc Philipp Schulze and others published Case Study: University offshore campuses and the regional development of Iskandar Malaysia | Find, read and cite all the research ...
Based on an urban greenwashing framework developed in this thesis and using Iskandar Malaysia project as a case study, this thesis operationalizes and quantifies (i) the environmental destruction; (ii) the promised greenness; and (iii) the price level of 38 target residential projects within the Iskandar region, and empirically confirmed ...
This case study is to identify and measure the outcome of Iskandar Malaysia's capital investment decision in Teluk Bahang, Johor. Iskandar Malaysia was founded in 2006 in Southern Peninsular Malaysia as an ambitious economic effort aimed for developing a significant and profitable development corridor in Malaysia's Johor area.
This article provides a critical overview on regional planning policies and implementation In Malaysia. Regional planning has implemented in Malaysia for almost 50 years, at the first 2 decades regional planning was tool for rapid growth (after independent) but this rapid mere economic and sectoral growth made regional disparities and it caused some part and ethnics be far from the development ...
Download Table | The Development in Iskandar Malaysia from publication: A study on the economics of local people -A case study of Iskandar Malaysia | This paper discusses the implication of ...
E-mail: [email protected] b Universiti Teknologi PETRONAS, 32610 Seri Iskandar, Perak Darul Ridzuan, Malaysia c ... this study is expected to be a solid initiation towards the development of future studies on the effects of hyperparameters in ML algorithms in reservoir characterization. ... A. N. Ahmed and A. El-Shafie, Rainfall ...