I'm having a hard time to believe this is caused by the Url Tracker.
I just checked one of our websites with 165k 404 entries, it's consuming 400MB of memory.
Even when I hit the back-office UI and navigate to different pages / opening entries the memory usage doesn't spike.

So I guess what I'm saying is that this is probably caused by something else.
I've explicitely chosen to store every 404 entry, because the referrer is stored too. It probably could be stored in a better way, but back then this was the best I could do.

Thanks for the quick response. Fair enough, I can't yet prove and I'm not 100% sure it's 301 URL Tracker. In fact, there may be some other confounding factor (e.g., SQL connection pool starvation). Here's a graph of memory and CPU usage over the last 24 hours:

Here are some notes for each numbered event:

1. Around 2PM, something happens that causes memory to climb steeply.
2. Around 5PM, I recycle the application pool in an attempt to lower the memory (shortly before that, I tinkered with Url Tracker dashboard, and each time it increased the memory usage).
3. Around 5:05PM, I do an iisreset (as the application pool recycle didn't seem to be reducing memory fast enough). This seems to lower memory usage.
4. Around 1:00AM, an automatic backup takes the machine offline, which lowers memory usage a bit.

I'll dig into the logs and ask some people what happened around 2PM, and that my provide some further clues. If i can find the time, I'll also get some memory profiling ready in case the issue happens again. My guess is that it has something to do with the fact that this site has about 60 Umbraco back office users. Perhaps many of them using the UrlTracker dashboard at the same time caused issues.

By the way, you could store referrers in another table and create a foreign key back to the main table with the URL's. That way, you could also keep a counter on the referrers rather than inserting a new record for each one.

Alternatively, you could just have the count on each row of the current table refer to the combination of old URL and referrer. If I group the data by old URL and referrer, there are about 27,000 results (i.e., about 10x less than 250,000):

FYI, the SQL connection pool starvation started right before 2PM yesterday (1:47PM), so that definitely seems related:

 2016-08-18 13:47:32,631 [P2996/D2/T29] ERROR umbraco.DataLayer.SqlHelper`1[[System.Data.SqlClient.SqlParameter, System.Data, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089]] - Error executing query SELECT * From UFPrevalueSources where id = @id
System.InvalidOperationException: Timeout expired.  The timeout period elapsed prior to obtaining a connection from the pool.  This may have occurred because all pooled connections were in use and max pool size was reached.
   at System.Data.ProviderBase.DbConnectionFactory.TryGetConnection(DbConnection owningConnection, TaskCompletionSource`1 retry, DbConnectionOptions userOptions, DbConnectionInternal oldConnection, DbConnectionInternal& connection)
   at System.Data.ProviderBase.DbConnectionInternal.TryOpenConnectionInternal(DbConnection outerConnection, DbConnectionFactory connectionFactory, TaskCompletionSource`1 retry, DbConnectionOptions userOptions)
   at System.Data.SqlClient.SqlConnection.TryOpenInner(TaskCompletionSource`1 retry)
   at System.Data.SqlClient.SqlConnection.TryOpen(TaskCompletionSource`1 retry)
   at System.Data.SqlClient.SqlConnection.Open()
   at System.Data.SqlClient.SqlConnection.Open()
   at Microsoft.ApplicationBlocks.Data.SqlHelper.ExecuteReader(String connectionString, CommandType commandType, String commandText, SqlParameter[] commandParameters)
   at umbraco.DataLayer.SqlHelpers.SqlServer.SqlServerHelper.ExecuteReader(String commandText, SqlParameter[] parameters)
   at umbraco.DataLayer.SqlHelper`1.ExecuteReader(String commandText, IParameter[] parameters)

I suspect this is due to a combination of factors, so perhaps it's only partially related to 301 URL Tracker.

Just came across another potential cause. I accidentally typed an "e" into the search box:

When I did that, the screen became lighter (to indicate an operation was in progress) and it pegged the CPU for 63 seconds. That seems a bit insane, and this is definitely an area that could use some improvement. I'm not sure that's what lead to the memory issues, but it may have been a factor (e.g., if it's opening a bunch of SQL queries that then eventually leads to connection pool starvation).

By the way, it seems that any key will trigger the search. If I don't touch the search box, and I press END (to scroll to the bottom of the page), that triggers a search.

Valid feedback and I know there's lots of room for improvement, but as you might know I am unable to maintain the project and I don't think I'm ever going to do that again.
As you might know some other people stepped up to add some funtionality in the core: http://issues.umbraco.org/issue/U4-8361
I was always willing to be a part of that and also mentioned that I wanted to help, but this happened behind my back and I feel kind of disappointed about it.
So it's a matter of lacking time, but also not wanting to spend more time on it anymore, sorry about that. I hope someone else will be able to release an update soon.

Understood. Thanks for letting me know.

I suppose the answer for now is to simply disable 404 tracking.

FYI, I have a new clue. Check this out: https://our.umbraco.org/forum/using-umbraco-and-getting-started/79722-cpunetwork-saturated-after-sorting-content-nodes

Basically, because of some inefficient code, sort operations on the site in question cause CPU/memory to shoot up. I may only appear that it's related to the 301 URL Tracker because 301 URL Tracker also has inefficiencies.

Not sure about that, but thought I'd let you know in case it helps anybody else.

I'm having a similar issue.. i have 301 Url Tracker v3.11 installed on a local instance of a website (running Umbraco 7.4.3), and it works perfectly.

However, on the pre-production server, navigating to the Url Tracker tab on the Content home page in Umbraco causes the app pool to start to consume massive amounts of RAM, until it crashes.

I know, 100%, that it is clicking on that tab that causes this behaviour - there's no doubt about it.

No errors are logged in the Umbraco log, and the only error displayed is an eventual timeout once the app pool has crashed.

Any ideas what's causing this? I tried to disable the Url Tracker (via urlTracker:trackingDisabled ) but no go.

Any other suggestions would be welcome as I'm almost ready to give up and find another solution.

Thanks heaps Greg

is working on a reply...