Enabled TPC backpressure with 256 pending requests limit

The back pressure messages you are observing are from TPC (Thread per Core) back pressure:


Example log:

INFO [main] 2020-12-12 23:20:46,067 EpollTPCEventLoopGroup.java:141 - Enabled TPC backpressure with 256 pending requests limit, remote multiplier at 5, global multiplier at 11

INFO [main] 2020-12-12 23:20:46,067 EpollTPCEventLoopGroup.java:146 - TPC extended backoff is enabled

INFO [main] 2020-12-12 23:20:46,067 TPC.java:138 - Created 11 epoll event loops.

INFO [main] 2020-12-12 23:20:46,073 TPC.java:153 - Created 2 TPC timers due to configured ratio of 5.

The TPC back pressure implementation is designed to avoid overloading the server with either client or replica requests (read/write/range), which could make the server unresponsive and/or lead to long garbage collections and out of memory errors. The setting for this is via tpc_pending_requests_limit which defaults to 256.

The setting affected by the back_pressure_enabled setting is used to restrict the network connections which are "on hold" from other nodes at the network level. This will trigger TPC back pressure if it is enabled AND "on hold" network connections exceed the setting value.

The "tpc_concurrent_requests_limit" and "tpc_pending_requests_limit" settings are set in the cassandra.yaml file but they are not included by default. They are currently not documented.

***