News:

Wondering if this will always be free?  See why free is better.

Main Menu

Handling Distributed Attacks

Started by InterestedForumUser, July 28, 2016, 10:37:06 PM

Previous topic - Next topic

InterestedForumUser

We are having distributed attacks for what seems like the last half a year.  We are getting a large number of guest users from random ips supposedly browsing the forum. Looking into their activities they are registering (nothing comes of this) and finding random temporally distant posts and looking at them. Our post history spans about 10 years and so is quite large.

Since they aren't picked up by our analytics and make up around 70% of our load I was wondering what options in SMF could help mitigate this kind of attack as they have been draining our server resources to almost unusable standards.

Kindred

#1
There really isn't anything that can be done...

You could ban the ip ranges at the host level or in htaccess...

However, it is more likely that what you are seeing is spammers or baidu bots, not actually ddns attacks.
Слaва
Украинi

Please do not PM, IM or Email me with support questions.  You will get better and faster responses in the support boards.  Thank you.

"Loki is not evil, although he is certainly not a force for good. Loki is... complicated."

InterestedForumUser

First off, thanks for taking the time to answer me and help!

Unfortunately they aren't really any particular range and are spread out. Would implementing a blacklist help with this then if its braidu bots / spammers or is there really just no hope aside from beefing up the server?

vbgamer45

Block some of the bad bots with .htaccess would help.
For my forums I put them behind a hardware firewall.
Community Suite for SMF - Take your forum to the next level built for SMF, Gallery,Store,Classifieds,Downloads,more!

SMFHacks.com -  Paid Modifications for SMF

Mods:
EzPortal - Portal System for SMF
SMF Gallery Pro
SMF Store SMF Classifieds Ad Seller Pro

a10

Maybe content scrapers? They'll go away after a while, but a plague, can hit 10.000's times. Example, search google for ahrefs bot, returns much useful info.

For some really overactive bots, am having fun picking out the worst, banning the ip (or ranges) for a while, and sending them away (to asklfjiowefliwdjfiwjeijwiefj.com or some real site that deserves visits, or back to their own site :O)

Edit security.php
http://www.simplemachines.org/community/index.php?topic=37005.msg271996#msg271996
There's also a mod for it.
2.0.19, php 8.0.23, MariaDB 10.5.15. Mods: Contact Page, Like Posts, Responsive Curve, Search Focus Dropdown, Add Join Date to Post.

InterestedForumUser

Yeah, I had thought about that, and I managed to ban a block of them a few months ago that brought my illegit user count down to 100 from something like 500. Still my performance is killing me though. Perhaps I should look into performance increases (which I have been to little success)

Quote from: a10 on July 29, 2016, 04:01:00 AM
Maybe content scrapers? They'll go away after a while, but a plague, can hit 10.000's times. Example, search google for ahrefs bot, returns much useful info.

For some really overactive bots, am having fun picking out the worst, banning the ip (or ranges) for a while, and sending them away (to hxxp:asklfjiowefliwdjfiwjeijwiefj.com [nonactive] or some real site that deserves visits, or back to their own site :O)

Edit security.php
http://www.simplemachines.org/community/index.php?topic=37005.msg271996#msg271996
There's also a mod for it.
Its been going on so long its hard to believe its scrapers, though we do have 10 years of posts so it could be. I'll make that edit; not a bad idea. Thanks!

InterestedForumUser

#6
As a side note, maybe I'm wrong about it being simply a matter of an attack. Here's my status.php
Connections per second: 42.3453
Kilobytes received per second: 327.8142
Kilobytes sent per second: 20612.5895
Queries per second: 1471.3518
Percentage of slow queries: 0.0012
Opened vs. Open tables:
(table_cache) 1409.1676 (should be <= 80)
Key buffer read hit rate:
(key_buffer_size) 0.0067 (should be <= 0.01)
Key buffer write hit rate:
(key_buffer_size) 0.4526 (should be <= 0.5)
Thread cache hit rate:
(thread_cache_size) 22.0323 (should be >= 30 )
Thread cache usage:
(thread_cache_size) 0.125 (should be >= 0.7 and <= 0.9)
Temporary table disk usage:
(tmp_table_size) 0.4916 (should be <= 0.5)
Sort merge pass rate:
(sort_buffer) 0.0001 (should be <= 0.001)
Query cache enabled:
(query_cache_type) 1 (should be >= 1 and <= 1)
Query cache miss rate:
(query_cache_limit) 0.6235 (should be <= 0.5)
Query cache prune rate:
(query_cache_size) 0.2399 (should be <= 0.05)

MySQL status
Aborted_clients: 649107
Aborted_connects: 49621
Access_denied_errors: 36726
Aria_pagecache_blocks_not_flushed: 1
Aria_pagecache_blocks_unused: 15735
Aria_pagecache_blocks_used: 15736
Aria_pagecache_read_requests: 25241705606
Aria_pagecache_reads: 5243040524
Aria_pagecache_write_requests: 4721854006
Aria_pagecache_writes: 3520788264
Aria_transaction_log_syncs: 21085
Binlog_commits: 0
Binlog_group_commits: 0
Binlog_group_commit_trigger_count: 0
Binlog_group_commit_trigger_lock_wait: 0
Binlog_group_commit_trigger_timeout: 0
Binlog_snapshot_file:
Binlog_snapshot_position: 0
Binlog_bytes_written: 0
Binlog_cache_disk_use: 0
Binlog_cache_use: 0
Binlog_stmt_cache_disk_use: 0
Binlog_stmt_cache_use: 0
Busy_time: 0.000000
Bytes_received: 382571747156
Bytes_sent: 24055684769860
Com_admin_commands: 3809835
Com_alter_db: 24
Com_alter_db_upgrade: 0
Com_alter_event: 0
Com_alter_function: 0
Com_alter_procedure: 0
Com_alter_server: 0
Com_alter_table: 33164
Com_alter_tablespace: 0
Com_analyze: 190
Com_assign_to_keycache: 0
Com_begin: 248169
Com_binlog: 0
Com_call_procedure: 65
Com_change_db: 42047290
Com_change_master: 0
Com_check: 1373
Com_checksum: 14
Com_commit: 278023
Com_create_db: 111
Com_create_event: 0
Com_create_function: 0
Com_create_index: 0
Com_create_procedure: 0
Com_create_role: 0
Com_create_server: 0
Com_create_table: 84925
Com_create_trigger: 0
Com_create_udf: 0
Com_create_user: 0
Com_create_view: 0
Com_dealloc_sql: 0
Com_delete: 5734338
Com_delete_multi: 12139
Com_do: 0
Com_drop_db: 67
Com_drop_event: 0
Com_drop_function: 0
Com_drop_index: 0
Com_drop_procedure: 0
Com_drop_role: 0
Com_drop_server: 0
Com_drop_table: 7286
Com_drop_trigger: 0
Com_drop_user: 91
Com_drop_view: 0
Com_empty_query: 31468
Com_enable_governor: 0
Com_enable_governor_reconn: 0
Com_enable_governor_lve: 18993
Com_enable_governor_reconn_lve: 1
Com_execute_sql: 0
Com_flush: 50432
Com_get_diagnostics: 0
Com_grant: 3656
Com_grant_role: 0
Com_ha_close: 0
Com_ha_open: 0
Com_ha_read: 0
Com_help: 0
Com_insert: 37812282
Com_insert_select: 254866
Com_install_plugin: 0
Com_kill: 12368
Com_load: 134
Com_lock_tables: 35514
Com_optimize: 4515
Com_preload_keys: 0
Com_prepare_sql: 0
Com_purge: 0
Com_purge_before_date: 0
Com_release_savepoint: 56793
Com_rename_table: 0
Com_rename_user: 0
Com_repair: 260
Com_replace: 1908766
Com_replace_select: 466
Com_reset: 0
Com_resignal: 0
Com_revoke: 2
Com_revoke_all: 0
Com_revoke_role: 0
Com_rollback: 32
Com_rollback_to_savepoint: 1698
Com_savepoint: 56870
Com_select: 1416430905
Com_set_option: 69066765
Com_show_authors: 0
Com_show_binlog_events: 0
Com_show_binlogs: 475
Com_show_charsets: 1797
Com_show_client_statistics: 0
Com_show_collations: 473
Com_show_contributors: 0
Com_show_create_db: 38
Com_show_create_event: 0
Com_show_create_func: 0
Com_show_create_proc: 0
Com_show_create_table: 28374
Com_show_create_trigger: 3
Com_show_databases: 869
Com_show_engine_logs: 0
Com_show_engine_mutex: 0
Com_show_engine_status: 1
Com_show_errors: 0
Com_show_events: 426
Com_show_explain: 0
Com_show_fields: 11260738
Com_show_function_status: 466
Com_show_grants: 9286
Com_show_index_statistics: 0
Com_show_keys: 334169
Com_show_master_status: 200
Com_show_open_tables: 0
Com_show_plugins: 40
Com_show_privileges: 0
Com_show_procedure_status: 466
Com_show_processlist: 79199
Com_show_profile: 0
Com_show_profiles: 0
Com_show_relaylog_events: 0
Com_show_slave_hosts: 0
Com_show_slave_status: 200
Com_show_status: 103946
Com_show_storage_engines: 431
Com_show_table_statistics: 0
Com_show_table_status: 46654
Com_show_tables: 9114587
Com_show_triggers: 3846
Com_show_user_statistics: 0
Com_show_variables: 58049
Com_show_warnings: 239
Com_shutdown: 0
Com_signal: 0
Com_start_all_slaves: 0
Com_start_slave: 0
Com_stmt_close: 24959639
Com_stmt_execute: 26846647
Com_stmt_fetch: 0
Com_stmt_prepare: 26846791
Com_stmt_reprepare: 8
Com_stmt_reset: 10221479
Com_stmt_send_long_data: 11857
Com_stop_all_slaves: 0
Com_stop_slave: 0
Com_truncate: 41499
Com_uninstall_plugin: 0
Com_unlock_tables: 35562
Com_update: 33015885
Com_update_multi: 41496
Com_xa_commit: 0
Com_xa_end: 0
Com_xa_prepare: 0
Com_xa_recover: 0
Com_xa_rollback: 0
Com_xa_start: 0
Compression: OFF
Connection_errors_accept: 0
Connection_errors_internal: 0
Connection_errors_max_connections: 0
Connection_errors_peer_address: 0
Connection_errors_select: 0
Connection_errors_tcpwrap: 0
Connections: 48260345
Cpu_time: 0.000000
Created_tmp_disk_tables: 47219062
Created_tmp_files: 954927
Created_tmp_tables: 96043287
Delayed_errors: 0
Delayed_insert_threads: 3
Delayed_writes: 545022
Empty_queries: 464910664
Enable_governor: 2
Executed_events: 0
Executed_triggers: 65
Feature_delay_key_write: 3018
Feature_dynamic_columns: 0
Feature_fulltext: 1882766
Feature_gis: 0
Feature_locale: 0
Feature_subquery: 32583449
Feature_timezone: 19949
Feature_trigger: 3674
Feature_xml: 0
Flush_commands: 25
Handler_commit: 175161941
Handler_delete: 17434544
Handler_discover: 0
Handler_external_lock: 0
Handler_icp_attempts: 110790079091
Handler_icp_match: 110150991932
Handler_mrr_init: 0
Handler_mrr_key_refills: 0
Handler_mrr_rowid_refills: 0
Handler_prepare: 0
Handler_read_first: 10098395
Handler_read_key: 272333134124
Handler_read_last: 1689940
Handler_read_next: 433746208433
Handler_read_prev: 2618543855
Handler_read_rnd: 9841854814
Handler_read_rnd_deleted: 18070012549
Handler_read_rnd_next: 385189036463
Handler_rollback: 8703
Handler_savepoint: 16147
Handler_savepoint_rollback: 1696
Handler_tmp_update: 194078345729
Handler_tmp_write: 69606136793
Handler_update: 188882031
Handler_write: 129270728
Innodb_available_undo_logs: 128
Innodb_background_log_sync: 1100972
Innodb_buffer_pool_bytes_data: 117489664
Innodb_buffer_pool_bytes_dirty: 11239424
Innodb_buffer_pool_dump_status: not started
Innodb_buffer_pool_load_status: not started
Innodb_buffer_pool_pages_data: 7171
Innodb_buffer_pool_pages_dirty: 686
Innodb_buffer_pool_pages_flushed: 39816494
Innodb_buffer_pool_pages_free: 815
Innodb_buffer_pool_pages_lru_flushed: 0
Innodb_buffer_pool_pages_made_not_young: 21974279977
Innodb_buffer_pool_pages_made_young: 117967643
Innodb_buffer_pool_pages_misc: 205
Innodb_buffer_pool_pages_old: 2667
Innodb_buffer_pool_pages_total: 8191
Innodb_buffer_pool_read_ahead: 229492324
Innodb_buffer_pool_read_ahead_evicted: 0
Innodb_buffer_pool_read_ahead_rnd: 0
Innodb_buffer_pool_read_requests: 52997735551
Innodb_buffer_pool_reads: 857919758
Innodb_buffer_pool_wait_free: 0
Innodb_buffer_pool_write_requests: 640108397
Innodb_checkpoint_age: 903901
Innodb_checkpoint_max_age: 80826164
Innodb_data_fsyncs: 26410538
Innodb_data_pending_fsyncs: 0
Innodb_data_pending_reads: 0
Innodb_data_pending_writes: 0
Innodb_data_read: 17944466165760
Innodb_data_reads: 1095271100
Innodb_data_writes: 56510694
Innodb_data_written: 1390429271552
Innodb_dblwr_pages_written: 39816494
Innodb_dblwr_writes: 3608614
Innodb_deadlocks: 6
Innodb_have_atomic_builtins: ON
Innodb_history_list_length: 1032
Innodb_ibuf_discarded_delete_marks: 166
Innodb_ibuf_discarded_deletes: 0
Innodb_ibuf_discarded_inserts: 4
Innodb_ibuf_free_list: 6141
Innodb_ibuf_merged_delete_marks: 7638631
Innodb_ibuf_merged_deletes: 900134
Innodb_ibuf_merged_inserts: 12258745
Innodb_ibuf_merges: 9394641
Innodb_ibuf_segment_size: 6143
Innodb_ibuf_size: 1
Innodb_log_waits: 0
Innodb_log_write_requests: 167056777
Innodb_log_writes: 12771290
Innodb_lsn_current: 2181977420271
Innodb_lsn_flushed: 2181977420271
Innodb_lsn_last_checkpoint: 2181976516370
Innodb_master_thread_active_loops: 1098442
Innodb_master_thread_idle_loops: 2530
Innodb_max_trx_id: 9173395485
Innodb_mem_adaptive_hash: 5576288
Innodb_mem_dictionary: 100330578
Innodb_mem_total: 137363456
Innodb_mutex_os_waits: 20631016
Innodb_mutex_spin_rounds: 759787384
Innodb_mutex_spin_waits: 68185646
Innodb_oldest_view_low_limit_trx_id: 0
Innodb_os_log_fsyncs: 12933947
Innodb_os_log_pending_fsyncs: 0
Innodb_os_log_pending_writes: 0
Innodb_os_log_written: 85640802816
Innodb_page_size: 16384
Innodb_pages_created: 2354178
Innodb_pages_read: 1095251224
Innodb_pages_written: 39816494
Innodb_purge_trx_id: 9173395419
Innodb_purge_undo_no: 0
Innodb_read_views_memory: 3728
Innodb_row_lock_current_waits: 18446744073709551615
Innodb_row_lock_time: 2937874
Innodb_row_lock_time_avg: 98
Innodb_row_lock_time_max: 28590
Innodb_row_lock_waits: 29779
Innodb_rows_deleted: 2999207
Innodb_rows_inserted: 31984367
Innodb_rows_read: 54829897579
Innodb_rows_updated: 146408104
Innodb_system_rows_deleted: 0
Innodb_system_rows_inserted: 0
Innodb_system_rows_read: 0
Innodb_system_rows_updated: 0
Innodb_s_lock_os_waits: 16351288
Innodb_s_lock_spin_rounds: 861259265
Innodb_s_lock_spin_waits: 51774064
Innodb_truncated_status_writes: 0
Innodb_x_lock_os_waits: 2336114
Innodb_x_lock_spin_rounds: 242224499
Innodb_x_lock_spin_waits: 17590758
Key_blocks_not_flushed: 0
Key_blocks_unused: 13
Key_blocks_used: 107163
Key_blocks_warm: 105601
Key_read_requests: 343683698099
Key_reads: 2314992419
Key_write_requests: 183098372
Key_writes: 82868133
Last_query_cost: 0.000000
Max_used_connections: 482
Memory_used: 945335480
Not_flushed_delayed_rows: 0
Open_files: 11833
Open_streams: 0
Open_table_definitions: 10000
Open_tables: 10000
Opened_files: 228959924
Opened_plugin_libraries: 0
Opened_table_definitions: 6027907
Opened_tables: 14091676
Opened_views: 252
Performance_schema_accounts_lost: 0
Performance_schema_cond_classes_lost: 0
Performance_schema_cond_instances_lost: 0
Performance_schema_digest_lost: 0
Performance_schema_file_classes_lost: 0
Performance_schema_file_handles_lost: 0
Performance_schema_file_instances_lost: 0
Performance_schema_hosts_lost: 0
Performance_schema_locker_lost: 0
Performance_schema_mutex_classes_lost: 0
Performance_schema_mutex_instances_lost: 0
Performance_schema_rwlock_classes_lost: 0
Performance_schema_rwlock_instances_lost: 0
Performance_schema_session_connect_attrs_lost: 0
Performance_schema_socket_classes_lost: 0
Performance_schema_socket_instances_lost: 0
Performance_schema_stage_classes_lost: 0
Performance_schema_statement_classes_lost: 0
Performance_schema_table_handles_lost: 0
Performance_schema_table_instances_lost: 0
Performance_schema_thread_classes_lost: 0
Performance_schema_thread_instances_lost: 0
Performance_schema_users_lost: 0
Prepared_stmt_count: 0
Qcache_free_blocks: 6591
Qcache_free_memory: 15149616
Qcache_hits: 855133835
Qcache_inserts: 476637878
Qcache_lowmem_prunes: 339847757
Qcache_not_cached: 84111557
Qcache_queries_in_cache: 15041
Qcache_total_blocks: 52470
Queries: 1738906967
Questions: 1676879013
Rows_read: 824807488803
Rows_sent: 43795623518
Rows_tmp_read: 268966727548
Rpl_status: AUTH_MASTER
Select_full_join: 2328832
Select_full_range_join: 523544
Select_range: 74472440
Select_range_check: 273163
Select_scan: 91701341
Slave_connections: 0
Slave_heartbeat_period: 0.000
Slave_open_temp_tables: 0
Slave_received_heartbeats: 0
Slave_retried_transactions: 0
Slave_running: OFF
Slaves_connected: 0
Slaves_running: 0
Slow_launch_threads: 0
Slow_queries: 1937611
Sort_merge_passes: 4496664
Sort_priority_queue_sorts: 31825868
Sort_range: 91399738
Sort_rows: 69094526771
Sort_scan: 74923560
Ssl_accept_renegotiates: 0
Ssl_accepts: 0
Ssl_callback_cache_hits: 0
Ssl_cipher:
Ssl_cipher_list:
Ssl_client_connects: 0
Ssl_connect_renegotiates: 0
Ssl_ctx_verify_depth: 0
Ssl_ctx_verify_mode: 0
Ssl_default_timeout: 0
Ssl_finished_accepts: 0
Ssl_finished_connects: 0
Ssl_server_not_after:
Ssl_server_not_before:
Ssl_session_cache_hits: 0
Ssl_session_cache_misses: 0
Ssl_session_cache_mode: NONE
Ssl_session_cache_overflows: 0
Ssl_session_cache_size: 0
Ssl_session_cache_timeouts: 0
Ssl_sessions_reused: 0
Ssl_used_session_cache_entries: 0
Ssl_verify_depth: 0
Ssl_verify_mode: 0
Ssl_version:
Subquery_cache_hit: 2289618
Subquery_cache_miss: 179736250
Syncs: 49703
Table_locks_immediate: 817108742
Table_locks_waited: 2210834
Tc_log_max_pages_used: 0
Tc_log_page_size: 0
Tc_log_page_waits: 0
Threadpool_idle_threads: 0
Threadpool_threads: 0
Threads_cached: 2
Threads_connected: 87
Threads_created: 2190435
Threads_running: 12
Uptime: 1139686
Uptime_since_flush_status: 1139686

MySQL variables
aria_block_size: 8192
aria_checkpoint_interval: 30
aria_checkpoint_log_activity: 1048576
aria_force_start_after_recovery_failures: 0
aria_group_commit: none
aria_group_commit_interval: 0
aria_log_file_size: 1073741824
aria_log_purge_type: immediate
aria_max_sort_file_size: 9223372036853727232
aria_page_checksum: ON
aria_pagecache_age_threshold: 300
aria_pagecache_buffer_size: 134217728
aria_pagecache_division_limit: 100
aria_pagecache_file_hash_size: 512
aria_recover: NORMAL
aria_repair_threads: 1
aria_sort_buffer_size: 268434432
aria_stats_method: nulls_unequal
aria_sync_log_dir: NEWFILE
aria_used_for_temp_tables: ON
auto_increment_increment: 1
auto_increment_offset: 1
autocommit: ON
automatic_sp_privileges: ON
back_log: 150
basedir: /usr
big_tables: OFF
binlog_annotate_row_events: OFF
binlog_cache_size: 32768
binlog_checksum: NONE
binlog_commit_wait_count: 0
binlog_commit_wait_usec: 100000
binlog_direct_non_transactional_updates: OFF
binlog_format: STATEMENT
binlog_optimize_thread_scheduling: ON
binlog_stmt_cache_size: 32768
bulk_insert_buffer_size: 8388608
character_set_client: latin1
character_set_connection: latin1
character_set_database: latin1
character_set_filesystem: binary
character_set_results: latin1
character_set_server: latin1
character_set_system: utf8
character_sets_dir: /usr/share/mysql/charsets/
collation_connection: latin1_swedish_ci
collation_database: latin1_swedish_ci
collation_server: latin1_swedish_ci
completion_type: NO_CHAIN
concurrent_insert: AUTO
connect_timeout: 15
datadir: /var/lib/mysql/
date_format: %Y-%m-%d
datetime_format: %Y-%m-%d %H:%i:%s
deadlock_search_depth_long: 15
deadlock_search_depth_short: 4
deadlock_timeout_long: 50000000
deadlock_timeout_short: 10000
debug_no_thread_alarm: OFF
default_master_connection:
default_regex_flags:
default_storage_engine: MyISAM
default_week_format: 0
delay_key_write: ON
delayed_insert_limit: 100
delayed_insert_timeout: 300
delayed_queue_size: 1000
div_precision_increment: 4
error_count: 0
event_scheduler: OFF
expensive_subquery_limit: 100
expire_logs_days: 0
external_user:
extra_max_connections: 1
extra_port: 0
flush: OFF
flush_time: 0
foreign_key_checks: ON
ft_boolean_syntax: + -><()~*:""&|
ft_max_word_len: 84
ft_min_word_len: 4
ft_query_expansion_limit: 20
ft_stopword_file: (built-in)
general_log: OFF
general_log_file: sng001.log
group_concat_max_len: 1024
gtid_binlog_pos:
gtid_binlog_state:
gtid_current_pos:
gtid_domain_id: 0
gtid_ignore_duplicates: OFF
gtid_seq_no: 0
gtid_slave_pos:
gtid_strict_mode: OFF
have_compress: YES
have_crypt: YES
have_dynamic_loading: YES
have_geometry: YES
have_openssl: YES
have_profiling: YES
have_query_cache: YES
have_rtree_keys: YES
have_ssl: DISABLED
have_symlink: YES
histogram_size: 0
histogram_type: SINGLE_PREC_HB
host_cache_size: 128
hostname: sng001.hawkhost.com
identity: 0
ignore_builtin_innodb: OFF
ignore_db_dirs:
in_transaction: 0
init_connect:
init_file:
init_slave:
innodb_adaptive_flushing: ON
innodb_adaptive_flushing_lwm: 10.000000
innodb_adaptive_hash_index: ON
innodb_adaptive_hash_index_partitions: 1
innodb_adaptive_max_sleep_delay: 150000
innodb_additional_mem_pool_size: 8388608
innodb_api_bk_commit_interval: 5
innodb_api_disable_rowlock: OFF
innodb_api_enable_binlog: OFF
innodb_api_enable_mdl: OFF
innodb_api_trx_level: 0
innodb_autoextend_increment: 64
innodb_autoinc_lock_mode: 1
innodb_buffer_pool_dump_at_shutdown: OFF
innodb_buffer_pool_dump_now: OFF
innodb_buffer_pool_dump_pct: 100
innodb_buffer_pool_filename: ib_buffer_pool
innodb_buffer_pool_instances: 8
innodb_buffer_pool_load_abort: OFF
innodb_buffer_pool_load_at_startup: OFF
innodb_buffer_pool_load_now: OFF
innodb_buffer_pool_populate: OFF
innodb_buffer_pool_size: 134217728
innodb_change_buffer_max_size: 25
innodb_change_buffering: all
innodb_checksum_algorithm: innodb
innodb_checksums: ON
innodb_cleaner_lsn_age_factor: high_checkpoint
innodb_cmp_per_index_enabled: OFF
innodb_commit_concurrency: 0
innodb_compression_failure_threshold_pct: 5
innodb_compression_level: 6
innodb_compression_pad_pct_max: 50
innodb_concurrency_tickets: 5000
innodb_corrupt_table_action: assert
innodb_data_file_path: ibdata1:12M:autoextend
innodb_data_home_dir:
innodb_disable_sort_file_cache: OFF
innodb_doublewrite: ON
innodb_empty_free_list_algorithm: backoff
innodb_fake_changes: OFF
innodb_fast_shutdown: 1
innodb_file_format: Antelope
innodb_file_format_check: ON
innodb_file_format_max: Antelope
innodb_file_per_table: ON
innodb_flush_log_at_timeout: 1
innodb_flush_log_at_trx_commit: 1
innodb_flush_method:
innodb_flush_neighbors: 1
innodb_flushing_avg_loops: 30
innodb_force_load_corrupted: OFF
innodb_force_recovery: 0
innodb_foreground_preflush: exponential_backoff
innodb_ft_aux_table:
innodb_ft_cache_size: 8000000
innodb_ft_enable_diag_print: OFF
innodb_ft_enable_stopword: ON
innodb_ft_max_token_size: 84
innodb_ft_min_token_size: 3
innodb_ft_num_word_optimize: 2000
innodb_ft_result_cache_limit: 2000000000
innodb_ft_server_stopword_table:
innodb_ft_sort_pll_degree: 2
innodb_ft_total_cache_size: 640000000
innodb_ft_user_stopword_table:
innodb_io_capacity: 200
innodb_io_capacity_max: 2000
innodb_kill_idle_transaction: 0
innodb_large_prefix: OFF
innodb_lock_wait_timeout: 50
innodb_locking_fake_changes: ON
innodb_locks_unsafe_for_binlog: OFF
innodb_log_arch_dir: ./
innodb_log_arch_expire_sec: 0
innodb_log_archive: OFF
innodb_log_block_size: 512
innodb_log_buffer_size: 8388608
innodb_log_checksum_algorithm: innodb
innodb_log_compressed_pages: ON
innodb_log_file_size: 50331648
innodb_log_files_in_group: 2
innodb_log_group_home_dir: ./
innodb_lru_scan_depth: 1024
innodb_max_bitmap_file_size: 104857600
innodb_max_changed_pages: 1000000
innodb_max_dirty_pages_pct: 75.000000
innodb_max_dirty_pages_pct_lwm: 0.001000
innodb_max_purge_lag: 0
innodb_max_purge_lag_delay: 0
innodb_mirrored_log_groups: 1
innodb_monitor_disable:
innodb_monitor_enable:
innodb_monitor_reset:
innodb_monitor_reset_all:
innodb_old_blocks_pct: 37
innodb_old_blocks_time: 1000
innodb_online_alter_log_max_size: 134217728
innodb_open_files: 10000
innodb_optimize_fulltext_only: OFF
innodb_page_size: 16384
innodb_print_all_deadlocks: OFF
innodb_purge_batch_size: 300
innodb_purge_threads: 1
innodb_random_read_ahead: OFF
innodb_read_ahead_threshold: 56
innodb_read_io_threads: 4
innodb_read_only: OFF
innodb_replication_delay: 0
innodb_rollback_on_timeout: OFF
innodb_rollback_segments: 128
innodb_sched_priority_cleaner: 19
innodb_show_locks_held: 10
innodb_show_verbose_locks: 0
innodb_simulate_comp_failures: 0
innodb_sort_buffer_size: 1048576
innodb_spin_wait_delay: 6
innodb_stats_auto_recalc: ON
innodb_stats_method: nulls_equal
innodb_stats_modified_counter: 0
innodb_stats_on_metadata: OFF
innodb_stats_persistent: ON
innodb_stats_persistent_sample_pages: 20
innodb_stats_sample_pages: 8
innodb_stats_traditional: ON
innodb_stats_transient_sample_pages: 8
innodb_status_output: OFF
innodb_status_output_locks: OFF
innodb_strict_mode: OFF
innodb_support_xa: ON
innodb_sync_array_size: 1
innodb_sync_spin_loops: 30
innodb_table_locks: ON
innodb_thread_concurrency: 0
innodb_thread_sleep_delay: 10000
innodb_tmpdir:
innodb_track_changed_pages: OFF
innodb_undo_directory: .
innodb_undo_logs: 128
innodb_undo_tablespaces: 0
innodb_use_atomic_writes: OFF
innodb_use_fallocate: OFF
innodb_use_global_flush_log_at_trx_commit: ON
innodb_use_native_aio: ON
innodb_use_stacktrace: OFF
innodb_use_sys_malloc: ON
innodb_version: 5.6.29-76.2
innodb_write_io_threads: 4
insert_id: 0
interactive_timeout: 28800
join_buffer_size: 1048576
join_buffer_space_limit: 2097152
join_cache_level: 2
keep_files_on_create: OFF
key_buffer_size: 134217728
key_cache_age_threshold: 300
key_cache_block_size: 1024
key_cache_division_limit: 100
key_cache_file_hash_size: 512
key_cache_segments: 0
large_files_support: ON
large_page_size: 0
large_pages: OFF
last_gtid:
last_insert_id: 0
lc_messages: en_US
lc_messages_dir:
lc_time_names: en_US
license: GPL
local_infile: OFF
lock_wait_timeout: 31536000
locked_in_memory: OFF
log_bin: OFF
log_bin_trust_function_creators: OFF
log_error: /var/lib/mysql/sng001.hawkhost.com.err
log_output: FILE
log_queries_not_using_indexes: OFF
log_slave_updates: OFF
log_slow_filter: admin,filesort,filesort_on_disk,full_join,full_scan,query_cache,query_cache_miss,tmp_table,tmp_table_on_disk
log_slow_rate_limit: 1
log_slow_verbosity:
log_warnings: 1
long_query_time: 1.000000
low_priority_updates: OFF
lower_case_file_system: OFF
lower_case_table_names: 0
master_verify_checksum: OFF
max_allowed_packet: 268435456
max_binlog_cache_size: 18446744073709547520
max_binlog_size: 1073741824
max_binlog_stmt_cache_size: 18446744073709547520
max_connect_errors: 10000
max_connections: 500
max_delayed_threads: 20
max_digest_length: 1024
max_error_count: 64
max_heap_table_size: 134217728
max_insert_delayed_threads: 20
max_join_size: 18446744073709551615
max_length_for_sort_data: 1024
max_long_data_size: 268435456
max_prepared_stmt_count: 16382
max_relay_log_size: 1073741824
max_seeks_for_key: 4294967295
max_sort_length: 1024
max_sp_recursion_depth: 0
max_tmp_tables: 32
max_user_connections: 25
max_write_lock_count: 4294967295
metadata_locks_cache_size: 1024
metadata_locks_hash_instances: 8
min_examined_row_limit: 0
mrr_buffer_size: 262144
multi_range_count: 256
myisam_block_size: 1024
myisam_data_pointer_size: 6
myisam_max_sort_file_size: 9223372036853727232
myisam_mmap_size: 18446744073709551615
myisam_recover_options: DEFAULT
myisam_repair_threads: 1
myisam_sort_buffer_size: 100663296
myisam_stats_method: nulls_unequal
myisam_use_mmap: OFF
net_buffer_length: 16384
net_read_timeout: 30
net_retry_count: 10
net_write_timeout: 60
old: OFF
old_alter_table: OFF
old_mode:
old_passwords: OFF
open_files_limit: 100000
optimizer_prune_level: 1
optimizer_search_depth: 62
optimizer_selectivity_sampling_limit: 100
optimizer_switch: index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_merge_sort_intersection=off,engine_condition_pushdown=off,index_condition_pushdown=on,derived_merge=on,derived_with_keys=on,firstmatch=on,loosescan=on,materialization=on,in_to_exists=on,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on,subquery_cache=on,mrr=off,mrr_cost_based=off,mrr_sort_keys=off,outer_join_with_cache=on,semijoin_with_cache=on,join_cache_incremental=on,join_cache_hashed=on,join_cache_bka=on,optimize_join_buffer_size=off,table_elimination=on,extended_keys=on,exists_to_in=on
optimizer_use_condition_selectivity: 1
performance_schema: OFF
performance_schema_accounts_size: -1
performance_schema_digests_size: -1
performance_schema_events_stages_history_long_size: -1
performance_schema_events_stages_history_size: -1
performance_schema_events_statements_history_long_size: -1
performance_schema_events_statements_history_size: -1
performance_schema_events_waits_history_long_size: -1
performance_schema_events_waits_history_size: -1
performance_schema_hosts_size: -1
performance_schema_max_cond_classes: 80
performance_schema_max_cond_instances: -1
performance_schema_max_digest_length: 1024
performance_schema_max_file_classes: 50
performance_schema_max_file_handles: 32768
performance_schema_max_file_instances: -1
performance_schema_max_mutex_classes: 200
performance_schema_max_mutex_instances: -1
performance_schema_max_rwlock_classes: 40
performance_schema_max_rwlock_instances: -1
performance_schema_max_socket_classes: 10
performance_schema_max_socket_instances: -1
performance_schema_max_stage_classes: 150
performance_schema_max_statement_classes: 186
performance_schema_max_table_handles: -1
performance_schema_max_table_instances: -1
performance_schema_max_thread_classes: 50
performance_schema_max_thread_instances: -1
performance_schema_session_connect_attrs_size: -1
performance_schema_setup_actors_size: 100
performance_schema_setup_objects_size: 100
performance_schema_users_size: -1
pid_file: /var/lib/mysql/sng001.hawkhost.com.pid
plugin_dir: /usr/lib64/mysql/plugin/
plugin_maturity: unknown
port: 3306
preload_buffer_size: 32768
profiling: OFF
profiling_history_size: 15
progress_report_time: 5
protocol_version: 10
proxy_user:
pseudo_slave_mode: OFF
pseudo_thread_id: 48260337
query_alloc_block_size: 8192
query_cache_limit: 1048576
query_cache_min_res_unit: 4096
query_cache_size: 134217728
query_cache_strip_comments: OFF
query_cache_type: ON
query_cache_wlock_invalidate: OFF
query_prealloc_size: 8192
rand_seed1: 0
rand_seed2: 0
range_alloc_block_size: 4096
read_buffer_size: 262144
read_only: OFF
read_rnd_buffer_size: 524288
relay_log:
relay_log_index:
relay_log_info_file: relay-log.info
relay_log_purge: ON
relay_log_recovery: OFF
relay_log_space_limit: 0
replicate_annotate_row_events: OFF
replicate_do_db:
replicate_do_table:
replicate_events_marked_for_skip: replicate
replicate_ignore_db:
replicate_ignore_table:
replicate_wild_do_table:
replicate_wild_ignore_table:
report_host:
report_password:
report_port: 3306
report_user:
rowid_merge_buff_size: 8388608
rpl_recovery_rank: 0
secure_auth: OFF
secure_file_priv:
server_id: 0
skip_external_locking: ON
skip_name_resolve: OFF
skip_networking: OFF
skip_replication: OFF
skip_show_database: OFF
slave_compressed_protocol: OFF
slave_ddl_exec_mode: IDEMPOTENT
slave_domain_parallel_threads: 0
slave_exec_mode: STRICT
slave_load_tmpdir: /dev/shm
slave_max_allowed_packet: 1073741824
slave_net_timeout: 3600
slave_parallel_max_queued: 131072
slave_parallel_threads: 0
slave_skip_errors: OFF
slave_sql_verify_checksum: ON
slave_transaction_retries: 10
slave_type_conversions:
slow_launch_time: 2
slow_query_log: ON
slow_query_log_file: sng001-slow.log
socket: /var/lib/mysql/mysql.sock
sort_buffer_size: 524288
sql_auto_is_null: OFF
sql_big_selects: ON
sql_buffer_result: OFF
sql_log_bin: ON
sql_log_off: OFF
sql_mode:
sql_notes: ON
sql_quote_show_create: ON
sql_safe_updates: OFF
sql_select_limit: 18446744073709551615
sql_slave_skip_counter: 0
sql_warnings: OFF
ssl_ca:
ssl_capath:
ssl_cert:
ssl_cipher:
ssl_crl:
ssl_crlpath:
ssl_key:
storage_engine: MyISAM
stored_program_cache: 256
sync_binlog: 0
sync_frm: ON
sync_master_info: 0
sync_relay_log: 0
sync_relay_log_info: 0
system_time_zone: SGT
table_definition_cache: 10000
table_open_cache: 10000
thread_cache_size: 16
thread_concurrency: 10
thread_handling: one-thread-per-connection
thread_pool_idle_timeout: 60
thread_pool_max_threads: 500
thread_pool_oversubscribe: 3
thread_pool_size: 24
thread_pool_stall_limit: 500
thread_stack: 294912
time_format: %H:%i:%s
time_zone: SYSTEM
timed_mutexes: OFF
timestamp: 1469827629.647772
tmp_table_size: 134217728
tmpdir: /dev/shm
transaction_alloc_block_size: 8192
transaction_prealloc_size: 4096
tx_isolation: REPEATABLE-READ
tx_read_only: OFF
unique_checks: ON
updatable_views_with_limit: YES
use_stat_tables: NEVER
userstat: OFF
version: 10.0.25-MariaDB-cll-lve
version_comment: MariaDB Server
version_compile_machine: x86_64
version_compile_os: Linux
version_malloc_library: system
wait_timeout: 28800
warning_count: 0


So far to combat this on the side of helping performance I've done a few things.

  • Moved Avatar Directory
  • Increased Cache to Level 3 (which solved the issue for the forum index but nothing else)
  • Disabled Search for Guests
  • Cloudflare
  • Removed Old Avatars
  • Disabled Search for Guests
  • Enabled Memcache
  • Converted various tables to InnoDB
  • Disabled GZIP compression
  • Disabled most features (Calendar etc)
  • Disabled Recent Posts on Board Index
  • Database Driven Sessions
  • Replaced various image requests with font-awesome requests since it was already loaded
  • Pruned 40k+ zero post members
  • Tuned Search results to 30 per page and 300 max
  • Verified Text fields were not in use
  • Installed ZBBlock and uninstalled it due to it targeting a bunch of legitimate users (including myself)
  • Disabled Hostname lookups
  • Started building Large Custom Index (been running for about 2 days and counting of my waking hours clicking the button like its LOST


This has brought the TTFB down on all the secondary files loaded (images what not) but I'm still getting huge TTFB on the first request. I have managed to bring out CPU usage down from hitting the host limit to being around 0-40%.

Kindred

Almost everything you did there has absolutely no, or at the most, minimal effect
Слaва
Украинi

Please do not PM, IM or Email me with support questions.  You will get better and faster responses in the support boards.  Thank you.

"Loki is not evil, although he is certainly not a force for good. Loki is... complicated."

LiroyvH

Are you on a dedicated server with root access?
If so, have you considered for example throttling the amount of new connections in x secs/requests per IP/sec and maximum connections per IP? Also putting nginx in front might help you.

That's just some starters.
There is more that you can do on the server side than on the SMF side... :)
((U + C + I)x(10 − S)) / 20xAx1 / (1 − sin(F / 10))
President/CEO of Simple Machines - Server Manager
Please do not PM for support - anything else is usually OK.

InterestedForumUser

Unfortunately, this is a managed server. I pushed for VPS, but it ultimately was not up to me. Now that I've gotten the cpu requests down a bit I should be able to contact the server again and see if I can make some headway. I've also looked into various dev solutions such as programming up a simple honey pot.

shawnb61

I recently had a few users doing this by accident - many downloads, specific topics..   They were registered & answered a note.   Turns out they were using Safari & had those topics on a "reading list".   Some glitch must've happened, and Safari was continually trying to download the same topics over and over.    Once they removed the articles from the reading list, it was corrected.

If honeypot says they're suspicious, block them.   (I've had several of false positives on honeypot, though....)

Look up the ip address on whois.

If the ip address is tied to a registered user, send them a note.

If the traffic is a problem, shut them down, at least temporarily.   But learn whatever you can about them before doing so .   





Address the process rather than the outcome.  Then, the outcome becomes more likely.   - Fripp

shawnb61

I did find that robots.txt helped also.  Most bots apparently honor it.   

Address the process rather than the outcome.  Then, the outcome becomes more likely.   - Fripp

Advertisement: