Both online shoppers and e-commerce retailers benefit from price aggregation platforms that reduce searching costs for consumers and marketing expenses for businesses. The business model of Price Comparison Sites requires customers to frequently revisit their site. Therefore, they offer a range of services that support users in all pre-purchase stages of the buying process. A frequently used offer is the price alert service that notifies users when a customer-specific threshold price is reached. However, users are not assisted with configuring this service to find the best trade-off between waiting time and the amount of the defined saving. We use a large data set with 110,230 daily price observations for electronic consumer goods to develop a method that predicts when price alarms are triggered. The presented algorithm combines approaches from multiple fields and extends time series forecasting methodologies with a bootstrapped forecasting ensemble to generate various price development scenarios. We systematically reduce the uncertainty in the bootstrapped path space by dynamically calibrating a customised decision criterion and generate 62,061 automated predictions. Our proposed approach not only outperforms the benchmark forecasting models significantly in terms of accuracy but also produces precise estimates in cases where traditional approaches fail.