Machine Learning Model for Combined Peace River and Caloosahatchee River (SWFL Only)

1. Machine Learning Model (SWFL)¶

1.1 Data Loading and Preprocessing (SWFL)¶

The merged dataset is loaded and a binary target variable is created to indicate the presence of a Karenia brevis bloom. A bloom event is defined based on whether cell concentrations exceed 100,000 cells/L, with the target shifted one week ahead to support forecasting.

1.2 Lag Feature Engineering (SWFL)¶

Lagged variables and rolling averages are generated for key predictors such as nutrient concentrations, discharge, and ocean conditions. These temporal features provide context on recent environmental trends that may influence bloom dynamics.

1.3 Feature Selection (SWFL)¶

Relevant features, including both current and lagged environmental variables, are selected for use in the machine learning model. These features represent potential drivers of bloom events across both the Peace River and Caloosahatchee River systems.

1.4 Train-Test Split and Scaling (SWFL)¶

The dataset is split chronologically into training and testing sets to preserve temporal structure. A RobustScaler is applied to normalize the feature distributions and reduce sensitivity to outliers commonly present in hydrological and ecological data.

1.5 Model Training (SWFL)¶

A Random Forest classifier is trained on the scaled training set and evaluated on the test set. Model performance is assessed using classification metrics to determine predictive accuracy for one-week-ahead bloom conditions.

1.6 Results Summary (SWFL)¶

The Random Forest classifier demonstrated strong performance in predicting Karenia brevis bloom activity one week in advance. The model achieved an overall accuracy of 90% and a balanced accuracy of 85.5%, indicating reliable performance across both bloom and non-bloom classes. For bloom prediction (≥100,000 cells/L), the model achieved a precision of 90% and a recall of 75%, with an F1-score of 0.82. This F1-score reflects the harmonic mean of precision and recall, offering a balanced measure of the model’s ability to minimize both false positives and false negatives — which is critical in ecological forecasting where both missed bloom events and false alarms carry consequences. Non-bloom weeks were identified with particularly high reliability, achieving an F1-score of 0.93. These results confirm that the inclusion of Caloosahatchee features — now properly scaled — alongside atmospheric drivers enhances the model’s ability to forecast red tide dynamics in Southwest Florida.

2. Evaluation (SWFL)¶

2.1 Confusion Matrix (SWFL)¶

Confusion Matrix: Random Forest Performance on Karenia brevis Bloom Forecasting¶

The confusion matrix visually represents the performance of your machine learning classification model—in this case, the Random Forest classifier used to predict Karenia brevis bloom severity. A confusion matrix summarizes the number of correct and incorrect predictions, broken down by each class (in this case, “No Bloom” and “Bloom”).

The confusion matrix has four key components:

1. True Negative (TN) = 172

   • The model correctly predicted “No Bloom” when there was actually no bloom.

2. False Positive (FP) = 7

   • The model predicted “Bloom” when there was actually no bloom (a false alarm).

3. False Negative (FN) = 20

   • The model predicted “No Bloom” when there was actually a bloom (a missed bloom event).

4. True Positive (TP) = 60

   • The model correctly predicted “Bloom” when there was actually a bloom.

Key Metrics Derived from the Confusion Matrix:¶

1. Accuracy – Measures the overall percentage of correct predictions:

   $$\text{Accuracy} = \frac{TP + TN}{TP + TN + FP + FN} = \frac{60 + 172}{60 + 172 + 7 + 20} = \frac{232}{259} = 0.90 \quad (90\%)$$

   (1)

2. Precision – Measures how many of the predicted positive cases (blooms) were actually correct:

   $$\text{Precision} = \frac{TP}{TP + FP} = \frac{60}{60 + 7} = 0.90 \quad (90\%)$$

   (2)

• High precision means fewer false positives (the model doesn’t incorrectly predict blooms very often).

3. Recall (Sensitivity or True Positive Rate) – Measures how many of the actual positive cases (blooms) were correctly identified:

   $$\text{Recall} = \frac{TP}{TP + FN} = \frac{60}{60 + 20} = 0.75 \quad (75\%)$$

   (3)

• High recall means the model catches most of the bloom events.

4. F1-Score – The harmonic mean of precision and recall; it balances the trade-off between precision and recall:

   $$\text{F1} = 2 \cdot \frac{\text{Precision} \cdot \text{Recall}}{\text{Precision} + \text{Recall}} = 2 \cdot \frac{0.90 \cdot 0.75}{0.90 + 0.75} = 0.82 \quad (82\%)$$

   (4)

• A high F1-score means the model achieves a good balance between precision and recall.

5. Support – The number of actual samples in each class:

   • No Bloom: 179 total instances

   • Bloom: 80 total instances

Summary of Model Performance:¶

• The model correctly predicted 90% of the test samples (accuracy).

• It had a high precision of 90%, meaning it rarely issued false bloom alerts.

• The recall of 75% indicates that the model detected most bloom events but still missed 20 cases (false negatives).

• The F1-score of 82% reflects that the model maintains a good balance between precision and recall.

Conclusion:¶

• The model is good at predicting both bloom and no-bloom events.

• The high recall (75%) is critical for bloom forecasting because catching bloom events is more important than avoiding false alarms.

• The low number of false positives (7) suggests the model isn’t prone to issuing unnecessary warnings.

• The main area for improvement would be increasing the recall by reducing false negatives—perhaps through additional feature engineering or adjusting the decision threshold.

2.2 Precision-Recall Curve (SWFL)¶

This is a Precision-Recall (PR) Curve, which illustrates the trade-off between precision and recall at different classification thresholds for the Random Forest model trained on the combined Southwest Florida dataset (Peace River + Caloosahatchee River).

What the PR Curve Represents:¶

• Precision (Y-axis) = The proportion of predicted positive cases that are truly positive:

  $$\text{Precision} = \frac{TP}{TP + FP}$$

  (5)

• Recall (X-axis) = The proportion of actual positive cases that are correctly predicted:

  $$\text{Recall} = \frac{TP}{TP + FN}$$

  (6)

The PR curve evaluates the model’s performance across a range of thresholds, showing how precision changes as recall increases. High recall means the model captures more true blooms but may increase false positives, lowering precision. Conversely, high precision means fewer false alarms but may miss some bloom events.

In this model, the classifier maintains high precision (~0.90) even as recall increases toward 0.75 — reflecting a strong ability to identify true bloom events while minimizing false positives. A drop in precision occurs at very high recall levels (>0.80), indicating that attempts to capture every bloom come at the cost of issuing more false alarms — a common trade-off in environmental risk prediction.

This curve is consistent with the model’s F1-score of 82% and balanced accuracy of 85.5%, both of which reflect strong overall predictive power. The curve suggests that the model strikes an effective balance between sensitivity and specificity under standard thresholding. Depending on monitoring priorities, the threshold can be adjusted to favor higher recall (e.g., for early warning) or higher precision (e.g., to reduce false alerts). The F1-score or the area under the PR curve (PR AUC) can guide selection of an optimal decision threshold.

2.3 Feature Importance Plot (SWFL)¶

The plot from the Random Forest Classifier shows the relative contribution of each input feature to predicting Karenia brevis bloom severity. Feature importance reflects how much each feature reduces classification error, based on improvements in the Gini index across all trees.

Karenia brevis cell counts (kb) are by far the most influential feature, emphasizing the strong autocorrelation of bloom persistence. Lagged values (kb_prev1, kb_prev2) also rank highly, reinforcing the importance of short-term bloom memory in forecasting future conditions.

Discharge dynamics from both river systems are influential, particularly the 4-week average discharge (discharge_4w_avg) and lagged Peace River discharge (peace_discharge_prev1). Nutrient inputs from the Peace and Caloosahatchee Rivers also contribute meaningfully — including peace_TN_prev1, caloosa_TN_prev1, and caloosa_total_phosphorus — supporting the hypothesis that nutrient availability is a key short-term driver of bloom formation.

Oceanographic variables like zos (sea surface height anomaly) show moderate importance, while salinity ranks lower. Wind speed and direction show limited influence, likely reflecting their more variable and indirect effects on bloom development. These results confirm that bloom persistence, river discharge, and nutrient inputs are the dominant short-term factors in bloom severity across Southwest Florida, while atmospheric and ocean conditions contribute secondary effects such as transport and dispersion.

2.4 Partial Dependence Plots (SWFL)¶

Partial dependence plots (PDPs) illustrate the effect of river discharge and nutrient concentrations from both the Peace River and Caloosahatchee River on Karenia brevis bloom predictions, while holding other variables constant. The y-axis represents the predicted bloom probability, and the x-axis shows the range of each feature.

For both peace_discharge and caloosa_discharge, the model predicts increasing bloom probability as discharge rises from low to moderate levels, with the relationship flattening or slightly declining at higher flows. This pattern suggests that moderate freshwater inflow supports bloom development—potentially by delivering nutrients—but that the benefit diminishes or reverses at high discharge levels due to dilution or hydrodynamic flushing.

For nutrients, both peace_TN and caloosa_total_nitrogen show strong positive effects on bloom probability as concentrations increase from low to moderate levels. Beyond approximately 2–3 mg/L, the curves flatten or dip slightly, indicating diminishing influence or a potential nitrogen threshold above which blooms are no longer nutrient-limited.

peace_TP and caloosa_total_phosphorus follow similar nonlinear patterns: bloom probability increases steeply with rising phosphorus concentrations at low levels, but both curves exhibit a plateau or gradual decline beyond 1.0–1.5 mg/L. This may reflect ecological nutrient ratios shifting out of balance or reaching saturation levels beyond which additional phosphorus has little effect or becomes limiting in relation to nitrogen.

The clustering of black tick marks at the lower ends of the x-axes indicates that the majority of training data fall in these ranges, reinforcing that model predictions are most reliable there. These patterns collectively confirm that both river discharge and nutrient inputs are important short-term drivers of bloom risk, with nonlinear ecological responses consistent with threshold triggering, nutrient saturation, and hydrodynamic transport.

2.5 Actual Vs. Predicted Bloom (SWFL)¶

This figure shows the actual vs. predicted Karenia brevis bloom status over time for the combined Southwest Florida (SWFL) model. The model’s predictions (red dashed line) align closely with actual observed bloom events (blue line), demonstrating strong temporal agreement. Predicted bloom phases generally match the correct time windows, capturing both the onset and duration of bloom episodes in most cases.

The figure shows mild overprediction in select intervals — where predicted blooms persist slightly longer than observed. Occasional early or lagging activations also appear, but the model consistently captures the main bloom patterns with high fidelity.

These results suggest that the model continues to learn the temporal structure of blooms effectively across both river systems, with relatively few false negatives and a tendency toward conservative (slightly extended) bloom forecasting — which may be preferable in ecological monitoring contexts.

2.6 Learning Curve (SWFL)¶

The learning curve illustrates how model accuracy evolves as the size of the training set increases. The x-axis represents the number of training samples, while the y-axis shows model accuracy.

The training score (blue line) remains consistently at 1.00 across all training sizes, indicating that the Random Forest model fits the training data perfectly. While this demonstrates strong memorization of the training set, it also signals potential overfitting, as perfect accuracy on training data is rarely sustainable in practice.

The validation score (orange line) begins around 0.73 and improves steadily with larger training sets, reaching approximately 0.83–0.84. This suggests that increasing the amount of training data helps the model generalize better to unseen data, though performance gains diminish beyond a certain point — indicating the model may be approaching a plateau given the current features and configuration.

The persistent gap between the training and validation scores reflects a variance issue: the model is learning the training set too well but not fully translating that learning to the validation set. To address this, regularization techniques such as reducing tree depth, lowering the number of estimators, or limiting feature splits may help reduce overfitting. Alternatively, exploring simpler models (e.g., Logistic Regression or SVM) or enhancing the feature set could improve generalization.

Overall, the model performs well on validation data but shows room for refinement to reduce the training-validation gap and enhance robustness.

2.7 Pair Plot (SWFL)¶

The pair plot below shows the relationships and distributions among Karenia brevis cell concentrations (kb), river discharge, and nutrient levels from both the Peace River and Caloosahatchee River.

1. All discharge and nutrient variables exhibit strong right-skewed distributions, with most values concentrated at the low end and occasional high outliers. This reflects episodic nutrient loading and discharge events.

2. While no strong linear correlations are observed between kb and the environmental variables, cluster patterns suggest that higher bloom concentrations tend to occur in conjunction with moderate-to-high nutrient levels and discharge events.

3. Both rivers show similar distribution patterns in total nitrogen and total phosphorus, supporting the idea that nutrient surges from either system can contribute to bloom conditions.

These patterns suggest that Karenia brevis bloom dynamics are likely influenced by complex, nonlinear interactions involving both rivers, with threshold effects and episodic drivers rather than simple linear relationships.

3. XGBoost Model (SWFL)¶

We implemented XGBoost (Extreme Gradient Boosting) as an alternative machine learning model to predict harmful algal bloom (HAB) events one week in advance. XGBoost is a high-performance, tree-based algorithm known for its ability to model complex, nonlinear relationships in structured tabular data. It builds decision trees sequentially, with each new tree correcting the errors of the previous one — making it particularly effective for classification tasks.

We selected XGBoost because:

• It handles imbalanced datasets well using the scale_pos_weight parameter.

• It often outperforms traditional methods like Random Forest in predictive accuracy when properly tuned.

• It allows for fine-grained control over learning behavior and regularization to reduce overfitting.

XGBoost Model Evaluation Summary (SWFL)¶

The XGBoost classifier demonstrated strong performance in predicting Karenia brevis bloom events one week in advance using the Southwest Florida (SWFL) dataset. The model achieved an overall accuracy of 86% and a balanced accuracy of 83.7%, indicating solid predictive capability across both bloom and non-bloom classes.

Class-Specific Performance:

• No Bloom:

  • Precision: 0.92

  • Recall: 0.88

  • F1-score: 0.90

  • Support: 236 samples

• Bloom:

  • Precision: 0.71

  • Recall: 0.79

  • F1-score: 0.75

  • Support: 87 samples

The classifier performs particularly well on non-bloom predictions, with high precision and recall. For bloom predictions, the model achieves a recall of 0.79, meaning it correctly detects 79% of true bloom events, which is critical for early-warning applications. The precision of 0.71 indicates that most predicted blooms are indeed true positives, though there is room to reduce false positives.

Overall Averages:

• Macro average F1-score: 0.83

• Weighted average F1-score: 0.86

Confusion Matrix Summary:

These results suggest that the XGBoost model captures bloom dynamics effectively, with balanced performance between sensitivity (recall) and precision. Although the model slightly underperforms compared to the best-performing Random Forest configuration, it still provides reliable and interpretable predictions. With further hyperparameter tuning or feature enhancement, XGBoost could be a viable alternative for operational forecasting.

3.1 Quick Hyperparameter Tuning with RandomizedSearchCV (SWFL)¶

To improve the performance of the XGBoost model, we performed a quick hyperparameter tuning using RandomizedSearchCV. This approach samples a subset of possible parameter combinations rather than evaluating all combinations exhaustively. We used 20 random trials and 3-fold cross-validation to identify a better-performing configuration.

The hyperparameters tested included tree depth, learning rate, number of estimators, and subsampling ratios. We maintained class imbalance handling by preserving the scale_pos_weight. The optimized model was then evaluated on the test set to assess whether it improved bloom prediction performance over the original configuration.

3.2 Summary XGBoost Before and After Hyperparameter Tuning (SWFL)¶

We compared the performance of the default XGBoost model to a tuned version optimized using RandomizedSearchCV. The tuned model outperformed the default in several key areas, especially in detecting harmful algal blooms (the minority class).

Key Improvements:

• Balanced Accuracy increased from 0.837 to 0.844

• Bloom Recall improved from 0.79 to 0.82, meaning fewer actual blooms were missed

• Bloom F1-score rose from 0.75 to 0.76, indicating better balance between precision and recall

• False negatives (missed blooms) dropped from 18 to 16

Although the number of false positives increased slightly (from 28 to 30), the trade-off is acceptable given the importance of early and accurate bloom detection.

Overall, hyperparameter tuning led to a more robust and effective XGBoost model for forecasting bloom events.

3.3 Summary Tuned XGBoost vs. Random Forest (SWFL)¶

We compared the performance of the tuned XGBoost model to the Random Forest model in predicting harmful algal blooms. The results show that Random Forest slightly outperformed XGBoost overall, but the tuned XGBoost model offered advantages in recall for the bloom class.

Performance Summary:

• Accuracy: Random Forest = 0.90, Tuned XGBoost = 0.86

• Balanced Accuracy: Random Forest = 0.855, Tuned XGBoost = 0.844

• Bloom Precision: Random Forest = 0.90, Tuned XGBoost = 0.70

• Bloom Recall: Tuned XGBoost = 0.82, Random Forest = 0.75

• Bloom F1-score: Random Forest = 0.82, Tuned XGBoost = 0.76

• False Negatives (missed blooms): Tuned XGBoost = 16, Random Forest = 20

• False Positives: Tuned XGBoost = 30, Random Forest = 28

Conclusion:

• The Random Forest model provides stronger overall performance with higher accuracy and precision, and fewer false positives, making it suitable for applications where avoiding false alarms is a priority.

• The tuned XGBoost model, however, offers higher recall for bloom events, which means it catches more actual blooms and produces fewer false negatives. This can be critical in ecological or public health settings where missing a bloom is more consequential than issuing a false alert.

The best model ultimately depends on the risk tolerance and monitoring goals—whether the priority is minimizing missed bloom events (XGBoost) or reducing false bloom predictions (Random Forest).

4. Support Vector Classifier (Linear Kernel) (SWFL)¶

We implemented a Support Vector Classifier (SVC) with a linear kernel to serve as a strong linear baseline model for predicting harmful algal blooms. The linear SVC attempts to find the best hyperplane that separates the two classes (bloom and no bloom) with the maximum margin.

This model is particularly useful because:

• It performs well on moderately sized datasets.

• It provides a clear, interpretable decision boundary.

• It allows direct inspection of feature weights (similar to logistic regression).

• It is less prone to overfitting than more flexible nonlinear models when the data is approximately linearly separable.

We used class_weight='balanced' to account for class imbalance and evaluated the model using balanced accuracy and F1-score, with special attention to performance on the bloom class. This approach helps determine whether a simpler linear model can perform competitively with more complex models like Random Forest and XGBoost.

4.1 Re-Split and Scaled the Data (SWFL)¶

We re-split the data using train_test_split to ensure that the input features (X) and target labels (y) have matching sample sizes. This step eliminates any inconsistencies that may have occurred during earlier processing.

We also applied StandardScaler to normalize the feature values, which is important for models like Support Vector Machines that are sensitive to the scale of input data. Scaling ensures that all features contribute equally to the model’s decision boundary.

4.2 Summary of Support Vector Classifier (Linear Kernel) (SWFL)¶

The Support Vector Classifier (SVC) with a linear kernel achieved an overall accuracy of 0.84 and a balanced accuracy of 0.74. While the model performs well at identifying non-bloom cases, it struggles to consistently detect bloom events.

Class-wise performance:

• No Bloom (class 0):

  • Precision: 0.84

  • Recall: 0.96

  • F1-score: 0.90

  • Interpretation: The model is highly effective at detecting non-bloom conditions.

• Bloom (class 1):

  • Precision: 0.83

  • Recall: 0.52

  • F1-score: 0.64

  • Interpretation: Although the model is fairly accurate when it predicts a bloom, it misses nearly half of the actual bloom events.

Summary:

Overall, the linear SVC model is more conservative — it favors predicting “No Bloom” and only predicts “Bloom” when it is highly confident. This results in relatively few false bloom alerts but a higher number of missed bloom cases. The trade-off may be unsuitable in environmental applications where early detection of harmful blooms is critical. A more recall-optimized model might be preferred in these cases.

5. Model Comparison (SWFL)¶

5.1 Summary of Model Performance Comparison (SWFL)¶

The bar chart compares the performance of three classification models: Random Forest, Tuned XGBoost, and Linear SVC. Each model was evaluated using key metrics including overall accuracy, balanced accuracy, and class-specific performance for detecting harmful algal blooms.

Key Insights:

• Random Forest delivers the strongest overall performance:

  • Accuracy: 0.90

  • Balanced Accuracy: 0.88

  • Bloom Precision: 0.88

  • Bloom Recall: 0.80

  • Bloom F1-score: 0.84

  • Conclusion: This model offers a reliable balance between detecting blooms and avoiding false alarms.

• Tuned XGBoost shows competitive performance:

  • Accuracy: 0.86

  • Balanced Accuracy: 0.86

  • Bloom Precision: 0.70

  • Bloom Recall: 0.85

  • Bloom F1-score: 0.77

  • Conclusion: This model sacrifices some precision to catch more actual bloom events, making it ideal for situations where missing a bloom is riskier than a false positive.

• Linear SVC lags behind in bloom detection:

  • Accuracy: 0.85

  • Balanced Accuracy: 0.75

  • Bloom Precision: 0.81

  • Bloom Recall: 0.55

  • Bloom F1-score: 0.66

  • Conclusion: While it avoids false positives, it misses many blooms, making it less suitable when detection sensitivity is critical.

Conclusion:

• Random Forest provides the most balanced and accurate performance overall.

• Tuned XGBoost is preferred when high bloom recall is prioritized, despite a drop in precision.

• Linear SVC may be appropriate for conservative forecasting, but its low recall limits utility in ecological risk management.

6. KB Probability as a Function of TN and TP (SWFL)¶

6.1 Peace River KB Probability as a Function of TN and TP (SWFL)¶

KB Bloom Probability as a Function of TN and TP

The figures below visualize the predicted probability of Karenia brevis bloom activity as a function of total nitrogen (TN) and total phosphorus (TP) concentrations in the Peace River and Caloosahatchee River. These contour plots were generated using a Random Forest Classifier, which was trained to predict bloom occurrence based on historical environmental data.

Each plot illustrates how bloom probability responds to different combinations of TN and TP:

Peace River

• The model predicts increased bloom probability at moderate TN and TP concentrations, particularly when TN ranges from 2–6 mg/L and TP from 1–2 mg/L.

• Beyond these ranges, especially at higher TN or TP values, predicted bloom probability does not increase substantially — likely reflecting limited training data at those extremes or biological thresholds in bloom formation.

• The banded patterns suggest non-linear interactions between TN and TP, consistent with ecological feedbacks and nutrient saturation effects.

Caloosahatchee River

• The Caloosahatchee shows a more concentrated region of high bloom probability at TN values between ~1.0 and 2.0 mg/L and TP values around 0.1 to 0.2 mg/L.

• Outside this nutrient window, predicted bloom probability sharply declines, reflecting either environmental constraints or fewer bloom events historically associated with low or high nutrient concentrations.

• This tighter clustering of bloom-prone conditions suggests that the Caloosahatchee system may be more sensitive to specific nutrient thresholds than the Peace River.

These plots underscore that while nutrient enrichment is a key driver of bloom risk, the response is non-linear and system-specific. Integrating both TN and TP from each watershed improves predictive modeling and supports more targeted nutrient management strategies in Southwest Florida.

6.2 Calooshatchee River KB Probability as a Function of TN and TP (SWFL)¶