Opened 16 months ago

Last modified 6 weeks ago

#2145 assigned enhancement

Enhancements to Tunnel Manager server throttler

Reported by: Reportage Owned by: zab
Priority: minor Milestone: undecided
Component: apps/i2ptunnel Version: 0.9.32
Keywords: tunnel manager, throttler Cc:
Parent Tickets:

Description

  • Add ban duration to Connection Limit section
  • Enhance proxy error messages when client is blocked to indicate why the block occurred and the duration of the block
  • If maximum number of concurrent connections is reached, provide a custom error to indicate this in the proxy error message eg. "The server is currently experiencing a high load and new connections are being rejected. Please try again in {configured period} minutes.", and add a visual indicator to server tunnel under the Local Tunnels section in the sidebar
  • Enhance /tunnels to indicate active client blocks per server tunnel with client id, expiry time, total number of blocks enforced, and option to remove the block on any given client or add client to access blacklist; enable ajax refresh on /tunnels to ensure data is current
  • Optionally log all blocks and implement console page (or add section to /logs) to display, and for repeat offenders above a certain, user-determined threshold, automatically add to access blacklist for a user-configured period (0 to permanently add to blacklist)

Subtickets (add)

Change History (10)

comment:1 Changed 16 months ago by zzz

  • Status changed from new to open

Interesting ideas, most aren't easy, either for implementation or for the UI, there's a lot of complexity here. Additionally, the needs are way different from site to site, depending on both normal traffic patterns and the nature of the abnormal traffic.

We're also talking about adding blocking functions to i2pcontrol, which is a different and maybe better approach than the OP.

There's also the possibility of adding headers to the response, e.g. Retry-After
https://stackoverflow.com/questions/3764075/retry-after-http-response-header-does-it-affect-anything

There's also the 429 response and associated non-standard headers
https://httpstatuses.com/429
https://stackoverflow.com/questions/16022624/examples-of-http-api-rate-limiting-http-response-headers

One big problem is that while our rate limit GUI is in i2ptunnel, the rate limiting and responses are done in streaming (except for POST throttling). Streaming doesn't know what is HTTP and what isn't, so it can't send a HTTP response. The only choices are drop, close, or reset. Streaming knows nothing about HTTP.

So again, i2ptunnel would have to somehow configure streaming with a custom response.

Most of this needs to be driven by the people that need it. It's easy to think up improvements but it's when your site is being attacked that the needs become clear. If OP is motivated by actual experience that would be good to know, and sorting the list into high/low priority would also be helpful.

comment:2 Changed 16 months ago by zzz

An HTTP response including backoff timeframe in either the headers or text is helpful to humans but it 1) increases outbound bandwidth when under load and 2) is also helpful to attacker bots. There's no obvious default that would work for all situations.

comment:3 Changed 16 months ago by Reportage

List in terms of priority:

  • Ban duration to Inbound Connection Limits section: low complexity, obviously missing from the UI. Does the Max Concurrent connections section also require a ban duration field?
  • Enhanced proxy error messages: At the very least, if a connection is rejected (dropped) by the i2p server, the client's proxy tunnel should offer a clearer explanation of the block, indicating that the server is actively refusing the connection. "The website could be temporarily unavailable" is self-evidently true and not much help. "The website is actively rejecting your connection attempt. This may indicate that the server is under high load and is temporarily refusing new connections, it is offline and is refusing all connections, or it has blocked your access. You may want to retry later."
  • An addition to the /tunnels page to indicate active blocks, combined with a local tunnels indicator to indicate that a server tunnel is actively blocking (eg. a different colored star)

The ability to manipulate actively blocked clients, a dynamic blacklist and more explanatory proxy error messages are much more involved and therefore of lower priority.

comment:4 Changed 16 months ago by zzz

Thanks.

Streaming limits are implemented differently than the POST limits in i2ptunnel.

We don't have a separate ban duration implemented in streaming. Rate limits are simply in minute, hour, and day buckets. The ban duration is for the remainder of the bucket life. We could make it more elaborate, at a cost of code complexity and more memory usage. But it's not just a UI change.

A UI to show blocks would be nice, but it might be better delivered via the i2pcontrol API.

I'm most interested in the proposal deliver a HTTP message from streaming to the clients. That would improve user experience for everybody, not just the site ops.

comment:5 Changed 16 months ago by zzz

Working on the HTTP message from streaming for 0.9.34.

comment:6 Changed 15 months ago by zzz

HTTP 429 response from streaming for HTTP Server tunnels, in 76a1aae9236b2bab52477961d78f8741542b5d8d 09.33-6.

None of the rest is likely for .34

comment:7 Changed 14 months ago by slumlord

It may be useful to have a hidden service usage graph with connections/POST/bandwidth/etc information which would help better inform the operator of the hidden service as to what constitutes a normal level of usage.

comment:8 Changed 14 months ago by zzz

Per-tunnel bandwidth graphs have always been available when full stats are enabled. Changed in be76ba2255fc684793e36605d449b5a4225c42ba 0.9.33-23-rc to enable w/o full stats.

comment:9 Changed 14 months ago by zzz

Add links to bandwidth graphs on /tunnels
In 93a1a96011a02ab6b12e4ef5370aef1144428a9b to be 0.9.34-3

comment:10 Changed 6 weeks ago by zzz

  • Owner set to zab
  • Status changed from open to assigned

Reassigning to zab for review as possible features for the new and extensible streaming filter subsystem.

Note: See TracTickets for help on using tickets.