Resolved -
Over the past days we have reduced the blocking of IP addresses to only block specific IP's instead of whole ranges of cloud services.
This is unfortunately the only way to stop abusive users that we cannot approach by other means.
Our thredds.met.no service has now several days without major outages, and we consider the situation to be stable.
If you suspect that you have been blocked, use the contact address thredds@met.no and explain what you are doing and from which IP you have been running.
We would prefer to maintain our open data policy, but that also requires users to respect our terms of service.
We would like to thank those users that have contacted us and that now have "deparallalized" their scripts or made efforts to avoid inefficient traffic.
We also thank you for your patience during the past outages and hope that we can continue to deliver meteorological and oceanographic data open and free in the future too.
Dec 5, 12:37 CET
Monitoring -
We have added more resources to the public servers behind thredds.met.no and partly removed IP blocking while we monitor the situation.
The main problem still remains that some users are issuing a large number of parallel data extractions, which overload our service. As long as those users are "hiding" behind cloud services, we reserve the right block those IP ranges. We will continue to add or remove IP blocking in the coming days as we monitor the situation.
Dec 3, 08:28 CET
Update -
We have started blocking of certain cloud services as an attempt to reduce the overload on thredds.met.no.
We remind users that parallel downloads is against the terms of service and will result in blocking.
Nov 29, 16:15 CET
Update -
We are still experiencing stability issues, and the most likely source are users that are downloading extracts from large datasets in parallel processes. However, some of these users are coming from Amazon or Azure cloud services, which makes it very hard to block them as the IP changes from day to day or even hour to hour.
Nov 27, 15:52 CET
Investigating -
We are experiencing intermittent outages on thredds.met.no due to overload of the public service.
We may have to block certain IP's from using the service unless the users refrain from issuing massive parallel requests.
Priority customers are not affected by this outage.
Nov 22, 15:42 CET