Plan | Player Analytics

Plan | Player Analytics 5.6 build 2820

5.6 build 2820
This build contains some bugfixes to things that were missed in last release. If you missed that, here's the change log for that
Special thanks to jhqwqmc for contribution to this update
Change log
  • Fixed a error in extension boolean storage when using SQLite. A minor bug, but spammed the console a bit.
  • Fixed Join Address not appearing on the /plan ingame command due to formatting error
  • jhqwqmc updated Simplified Chinese Locale (CN)
5.6 build 2816
This version brings a lot of bugfixes, such as Geolocation database download changes, and some new features, such as CSV export, whitelist bounce gathering and more placeholders.
Special thanks to jhqwqmc, Vankka, Beniro, liuzhen932 and SlimeDog for contributions to this update!
Change log
Data Gathering
  • Plan now downloads geolite database from playeranalytics.net - This is related to MaxMind reducing their daily download counts to 30/day by March 21st which breaks the geolocation database downloading. This means the geolocation database needs to be distributed via a server rather than from MaxMind directly. Currently Plan implicitly trusts that the webserver responding from geodb.playeranalytics.net is correct one. In future there will be a signature check added to ensure Plan is talking to correct server.
  • Fixed server crash when using SQLite and Plan tried to gather installed plugin list on the server thread.
Allowlist bounces
allowlist-bounces
Spigot, Sponge & Nukkit servers now gather whitelist bounces. There is a new tab on the server page under Online Activity which allows you to quickly see whose login has been blocked by whitelist, and who you have already allowed on the whitelist based on session data, so that you don't do unnecessary work allowing them again.
Visibility is controlled with web permission page.server.allowlist.bounce
Website
CSV Export in tables
export-demo
You can now export CSV from most tables. This is especially useful when combined with Query features, such as the click-and-drag in calendar for selecting data from specific timeframe.
Other
  • The react bundle is now built using Vite. It utilizes modules available in modern browsers. This sped up build times by 7 minutes and should also improve page loading speed.
  • Fixed some issues where reverse proxied Plan website loaded a blank page
  • Fixed issue where server players online graph was fetched for network page due to bad cache lookup
  • Server & Network Overview now uses frontend formatting for playtime and dates
  • Fixed issue where server ping table never loaded
  • Fixed server calendar being editable (You could drag blocks around rather than select)
  • Fixed ping graph not rendering on player page if there was a lot of points
Webserver
  • Added simple DDoS protection that limits requests if same IP requests same path multiple times. This type of DDoS has become more prevalent against cloudflare IPs, where a Go-http-client based bot requests / repeatedly and doesn't follow redirects. Because browsers follow redirects this doesn't affect regular users.
Database
  • Optimized how database handles transactions on disable. More unnecessary transactions are dropped
Fabric 1.20.4
  • Binero implemented Fabric 1.20.4 support, 1.20.3 is not supported this build onwards.
  • Fabric command registration was moved earlier to avoid confusion when /plan reload doesn't exist after failed enable.
Placeholders
  • Added network_ equivalent to all missing server_ placeholders
  • Added %plan_regular_players% and %plan_network_regular_players% placeholders
  • Added %plan_join_address% placeholder
  • Fixed off-by-one error in %plan_top_...% placeholders where 0 would be top 1 and 9 top 10. Now it matches documentation where 1 -> 1, and 10 -> 10. If you were compensating for this bug you may need to change %plan_top_..._0% to %plan_top_..._1% etc
Commands
  • /plan ingame now shows Last join address of the player.
Export
  • Fixed react files getting exported even when all export is disabled.
Extensions
  • Vankka updated DiscordSRV Extension to fix an incompatibility with new DiscordSRV version
  • @Conditional values are now removed when the value of the boolean changes in a way that unsatisfies the condition.
  • Fixed FastLogin Extension recording "Unknown" when behind a proxy server
  • Fixed Quests Extension support for Quests version 5, version 4 is no longer supported.
Locale
  • jhqwqmc updated Chinese locale (CN)
  • liuzhen932 updated Chinese locale (CN)
Web User Access-Control - Change log 5.6 build 2614

Hello! This version contains a hefty set of new features, bugfixes and usability improvements. Highlighted features include Web User Access-Control, Plugin Version History and 'Click to see Who'-features. There are a lot more new things so check out the change log.

Updating to this version modifies database schema
There is a guide for updating to this version here: https://github.com/plan-player-analytics/Plan/wiki/Migration-guide-from-Version-5.5-to-5.6

Special thanks to yu_solt, ToxiWoxi, Kopo, xlanyleeet, Jumala9163, ringoXD, Dreeam-qwq & jhqwqmc for their contributions to this update!

If you would like to support the project financially, please consider sponsoring the main developer.

Change Log
Web user access control (HTTPS required)
[IMG]

A more granular user access control has been requested for quite a long time. Now you can limit users to only see smaller parts of the website. This feature is documented in https://github.com/plan-player-analytics/Plan/wiki/Web-permissions though you will find some help embedded on the /manage page.

Migrating from old version with permission_level adds groups 'legacy_level_0' etc with permissions that match previous behavior, users are linked to them automatically.

In order to edit web permissions on the website you need to set at least one webuser as admin with /plan setgroup {username} admin.

In order to keep /plan register working, give your players 'plan.webgroup.{group_name}'-permission in your permission plugin (such as LuckPerms) for the group you want them to have. If player has none of these permissions they can't register at all.

Plugin Version History (HTTPS required)
[IMG]

Plan now gathers plugin versions on server start. Any modifications are timestamped. This can be useful when tracking performance impact of plugin updates. This data was possible to gather from all server platforms.

Seeing this data on the website requires HTTPS to be set up because some server admins may consider this data sensitive. Without HTTPS the data endpoint is disabled and the data is only visible in the database. After HTTPS is set up admins can control who sees this data through web permissions.

On network page it is possible to check plugin versions of any server on the network. This can help you keep up to date with your plugins.

Click to see Who
[IMG]

You can now click & drag on Server Calendar (and the new Network Calendar) to see who was playing on specific days the calendar data is about. On Geolocations Map you can click on a Country to see who has joined from that country.

These features utilize the existing Query features so it was relatively simple to implement. If you have more ideas where you would like to see who data is about, you can make a suggestions on Github

Website
New features

  • Player tables now show Average, Best and Worst Ping for all players
  • You can now choose visible columns on any table (such as Players tables and plugin tables.)
  • Network Calendar was added to Network Overview, similar to Server Calendar
  • Navigation button can now go to Plan Error Logs and Swagger Docs
  • The bigger features mentioned above
Improvements
  • Network Performance now disables TPS, Entities and Chunks buttons if there's only Proxy servers in the selected dataset.
  • Player page Plugins navigation now only shows servers that the player has data from.
  • Fixed some table text color issues, especially in Night Mode
  • Fixed issues with reverse-proxy https falling back to relative addresses in frontend when proxy-mode https was not in use. (Plan would assume address is http:// but browser would have https://, leading to different start for the address, which was interpret as incorrect address.)
Removal of old frontend files
With the React rewrite being complete an in use by most users, this version deletes the old frontend files from the jar. This reduces jar size.

If you still haven't migrated your html customizations this version does not load them anymore. Migration guide: https://github.com/plan-player-analytics/Plan/wiki/Html-Customization--migration-guide-to-React

  • Removed Plugin.Use_Legacy_Frontend setting.
  • Removed any code related to this setting, old behavior and the old web files.
  • Moved most html rendering from backend to the frontend.
  • DataTables did not support rendering React inside table headers so the library was removed and all functionality used by Plan rewritten in React. The visual differences should be minimal.
PageExtension API changes
  • You can now use web permissions with WebUser
  • Registering custom web permissions is possible either by overriding Resolver#usedWebPermissions, or through ResolverService#registerPermissions. There are two methods in ResolverService, one which grants existing groups the new permissions based on existing permission.
Webserver
  • Address of any webserver that is enabled can now be used by commands (Useful for users with 2 game servers without proxy)
  • Disabled X-Forwarded-For reverse-proxy warning temporarily since it was giving false positive warnings often.
  • Fixed out of date information appearing on the /player page due to HTTP caching
Endpoint changes
  • Added /v1/preferences and /v1/storePreferences endpoints that has some default formatting values coming from config.yml and user specific preferences if they have them set. Preferences UI is not yet implemented so for now this only returns the defaults.
  • Deprecated /v1/players endpoint, replaced with /v1/playersTable which gives data in raw format instead of formatted.
  • Added Group Management related endpoints /v1/webGroups, /v1/groupPermissions, /v1/permissions, /v1/saveGroupPermissions, /v1/deleteGroup (Only enabled with https)
  • Added Plugin history endpoint /v1/pluginHistory (Only enabled with https)
Database
  • Fixed join address truncation error on backup
  • Added tables plan_web_group, plan_web_group_to_permission, plan_web_permission, plan_web_user_preferences & plan_plugin_versions
  • plan_security column permission_level was removed and a foreign key column group_id added
  • Inactive player cleaning is now disabled on Game servers if Proxy server is in the database to avoid confusing situations where limit configured on Proxy server doesn't apply.
Spigot
  • Possibly fixed Ping not being gathered on Spigot 1.20+ servers
  • Dreeam-qwq fixed an error that occurred when Join Address didn't contain a port number.
Fabric
  • Fabric 1.20.2 support was implemented by Kopo. Older versions of fabric are no longer supported.
  • Plan now uses semantic version for fabric metadata
Sponge
  • ToxiWoxi fixed SpongeForge failing to load Plan due to unspecified LuckPerms dependency version in Plan
Locale
Some functionality was added to make life of translators easier.

  • Added setting which lists untranslated keys Plugin.Logging.Log_untranslated_locale_keys (default false). This lists anything that has default value so it may give some false positives.
  • Custom locale.yml file is now reloaded when it is modified.
  • Ukranian Locale (UK) added by xlanyleeet
  • Japanese Locale (JA) updated by yu_solt and Jumala9163
  • Simplified Chinese Locale (CN) updated by jhqwqmc
  • Translation of some data was moved to frontend so that it changes when you change the language.
Placeholders
  • Fixed off by one mistake in %plan_top_...% placeholders where 1 would give the 2nd highest and 10 nothing.
  • Added %plan_player_geolocation% placeholder
Extensions
  • Fixed BuyCraft not sorting dates correctly
  • Fixed an issue where rapid fire Vulcan AntiCheat violations would cause database exceptions.
  • Updated Lands API to newer version
  • Possibly fixed an issue where Litebans data was not being updated
  • Fixed issue where ViaVersion would show no data on network page
5.5 build 2461
This update contains bugfixes. This is probably the last version that will support 'Plugin.Legacy_Frontend' setting. Version 5.6 will remove the old frontend code so migrate your html customizations to React as soon as possible.
Special thanks to DrexHD, lis2a & ringoXD for contributions to this update.
Change log
Data gathering
  • AFK time is now updated when server shuts down to avoid AFK players from appearing active if they remain AFK during shutdown.
Fabric
  • DrexHD updated to support fabric 1.20
  • Added support for netty based socket addresses to possibly fix geolocation gathering issue.
  • Fixed plugin incompatibility with Geyser and other plugins with default mixin reference map (Thanks to onebeastcris for reporting this)
Website
  • Fixed permission level 2 redirect to /player/{uuid}
  • Fixed issues with Query page not allowing to change Registered between or Played between filter dates
  • Latest join address pie is now sorted by 'Most players first'
Database
  • Added experimental support for MariaDB 11. (MariaDB 11.0.2 has a data insertion bug, use 11.1.1 if possible). This is automatically enabled if MySQL driver fails to connect to the database.
  • Fixed issue with plugin groups query related to Vault on Query page
  • Fixed join address truncation error when join address was over 191 characters.
Locale
  • lis2a and ringoXD updated Japanese (JA) Locale
Change Log 5.5 build 2391

This is a feature packed update, as it brings support for multiple proxy servers, IP allowlist CIDR and Dynamic DNS support, Fabric 1.19.4, and lots and lots of improvements.

Special thanks to Kopo, WolverStones & inductor for contributions to this update!

Change log
Fabric 1.19.4
  • Kopo updated Fabric support to 1.19.4, older versions are no longer be supported by this version
  • Kopo fixed Gamemode change event not triggering
  • Fixed a dog killing something crashing the server when running older fabric version
Multi proxy support
Plan can now be installed on multiple proxy servers. This is useful for networks that divide their players between multiple proxy servers.
  • You can now name proxy servers with Server.ServerName config option (to distinguish in Performance tab)
  • Server.IP setting is no longer required to be set, since some proxy servers may have their webserver disabled. It's recommended to have only a single Plan webserver running.
  • If multiple proxy servers have their webserver enabled, the commands that have links link to one of them.
  • If all webservers are disabled, but export is enabled, the commands that have links link to one with export enabled.
  • Network online graph stacks the player online counts of all the proxies if RedisBungee is not used. Non-stacked version can be checked from Performance tab.
Known limitations: Online sessions/status are not synced on multiple proxies

IP Allowlist improvements
  • IP Allowlist now supports a lot more notations for easier use: CIDR, Wildcard, IPv6 with omitted zeros. See comment above the setting for examples.
  • IP Allowlist now supports dynamic DNS addresses. Add "dns:{address}" to the allowlist to have Plan resolve the IP address of the dynamic DNS. This may be useful for those without static IP addresses at home.
Database
  • MySQL databases using incorrect character set are now corrected to use utf8mb4 collate utf8mb4_general_ci upon startup. This should prevent some errors from popping up.
  • Fixed join address by day graph error if MySQL strict group by policy is enabled.
  • If Plan detects that two servers are storing data with the same UUID, a warning will be logged to console once every 30 minutes.
    • This usually happens if ServerInfoFile.yml is copied between servers during installation.
    • /plan info command now shows the ServerUUID of the server so that it is easier to debug which server it is
    • The detection looks if previous tps was stored < 30s ago, which suggests two servers are storing data as the same one
  • Add Database.MySQL.Max_Lifetime setting. This setting can be used if database connection keeps timing out
Website
  • Added option to disable registering new users Webserver.Security.Disable_registration (default false)
  • Improvements to network performance tab
    • Added some information why TPS, entities or chunks is not available if only proxy servers are selected
    • Reduced resolution of the data to minutes, so that all servers with data show up in the tooltip when hovering over the graph
    • Fixed server selector not allowing to change servers in some cases
    • Fixed the graph re-rendering when selecting different servers even though the data wasn't reloaded
    • Added warning if none of the selected servers have sent data over 30 days
  • Improvements to network server list
    • An icon is shown if server hasn't sent data for some time (Red triangle if no data in last 24h, Grey triangle if no data in last 7d, Grey archive icon if no data in last 30d)
  • /plan server command now gives link to /server/UUID instead of /server/Name
  • Fixed configured theme not applying by default
  • Extension data with colors or links now render properly
  • Fixed the logged in player's head image not loading on the top right
Locale
  • Custom locale files (locale.yml) are now updated with new translation lines whenever server starts
  • WolverStones updated Czech (CS) locale
  • inductor updated Japanese (JA) locale
Extensions
  • Fixed ExceptionInInitializerError in Extension construction causing Plan to not enable properly
  • Fixed exception related to LibertyBans Extension
  • Fixed exception related to Towny Extension
5.5 build 2307
This release has some new tools for analyzing player retention, as well as further improvements to the website.

Special thanks to TheLittle_Yang for contributing to this update.

Change log
Player Retention Analysis Graph
retention-graph-possibilities

This update adds a graph to network and server pages (Playerbase > Player Retention) that has options to:
  • Draw graphs at different time resolutions
  • Limit input data by time
  • Group players by register date or join address
  • Visualize player retention in different ways
    • Time since registration date
    • Playtime
    • Date
    • Cumulative player gain
    • Percentage / Player count / Stacked player count
Any and all combinations are allowed which allows extensive analysis of player retention. Help sections attempt to make the data understandable and show examples.

Data gathering
  • Attempt to correct register dates if they report a date in the year 1970 (Epoch millisecond 0 = Jan 1 1970).
Website
  • Any extension tables can now be sorted
  • Fixed Network > Servers Overview Quick view graph not loading
  • Fixed sidebar being transparent in nightmode on mobile
  • Fixed Activity index help modal not drawing the index function after certain x threshold
  • Fixed data not updating when switching from Server to Network page in cases where they shared same components (eg. Playerbase graphs, Join address graphs, etc)
  • Improved player overview card layout on mobile
  • Reduced font size on mobile when in portrait mode, so that more tables remain readable
Extensions
  • Fixed NPE when player joined and FloodGate had not loaded its API
Locale
  • TheLittle_Yang updated Traditional Chinese (CN) Locale
5.5 build 2272
This update enables the new React based frontend by default, speeds up loading various parts of the website, and fixes a couple of issues.
If you are using Html Customization, it is now possible to migrate to the new system, see https://github.com/plan-player-analytics/Plan/wiki/Html-Customization--migration-guide-to-React - You can use Plugin.Use_Legacy_Frontend setting until your migrations are complete.
Back up your config in case you need to revert to previous version. Updating should be easy, simply replace the jar.
Change log
React Frontend (Previously called Frontend BETA)
The frontend rewrite is complete, so the new frontend is now enabled by default. Plugin.Use_Legacy_Frontend (default: false) setting still allows using old frontend for a while until it's completely removed.
Here is a summary of improvements this brings if you have not participated in the Beta:
  • Faster loading time since less data requests are made at once
  • Improved mobile navigation
  • New features
    • Page navigation button for switching between pages
    • Switching language on the frontend
    • Join address tab
    • Visualizer switches for some graphs
    • Average players online data to Performance tabs
    • Interactive '?' help for Activity Index and New Player Retention (These were the two most common questions on how they work)
    • Redesigned Network > Servers tab
  • Easier to maintain and develop further
Changes from previous update:
  • Added a page navigation button that allows moving to different servers and other pages easily. This replaces the 'Back to main page' button. You can switch between the same page for two servers (eg. Move from Server 1 > Performance to Server 2 > Performance in one click)
    image
  • Improved mobile navigation. With the navigation button this should help mobile users a lot.
  • Interactive '?' help for Activity Index and New Player Retention (These were the two most common questions on how they work)
  • Fixed network server list saying "No servers installed" while servers were being loaded.
  • Fixed page translation issues
  • React was updated to version 18
  • Javascript APIs for extending the page programmatically were implemented https://github.com/plan-player-analytics/Plan/wiki/APIv5-PageExtension-API#javascript-api
  • Fixed join address data breaking the page when visualized as a table
  • Fixed issues of plugin cards overlapping when switching between plugins of two servers on player page
New feature: public_html
A new feature in the webserver allows hosting any web files on the Plan webserver. Please note that any files placed in the public_html folder (/plugins/Plan/public_html by default) can be read by anyone who knows the address to the webserver even if you have login enabled. The folder can be configured with Webserver.Public_html_directory setting.
The main purpose of this feature is to allow Html Customization of the React bundle https://github.com/plan-player-analytics/Plan/wiki/Html-Customization
Webserver
  • Implemented HTTP Caching: Browser will now cache some responses and avoid sending unnecessary data if it was already loaded. This can improve page loading times from multiple seconds to milliseconds.
  • Implemented public_html feature that allows hosting custom files from a configurable folder Webserver.Public_html_directory (default /plugins/Plan/public_html). This can be used to host http-challenge file for certbot and other files.
Database
  • PlayerTableRowPatch should no longer be re-applied all the time
  • Optimized server player table query: /server/players now loads much faster. Tested optimization: 4s -> 500ms: 8x improvement
  • Optimized server latest join addresses query: /sever/join_addresses now loads much faster. Tested optimization: 19s -> 150ms: 120x improvement
  • Optimized /v1/network/servers endpoint, got a 66% speed increase, so Servers tab on network page should load faster.
PlaceholderAPI
  • Unregister placeholder extension when Plan disables: this possibly fixes an issue where PlaceholderAPI would log errors when Plan disabled before PlaceholderAPI.
5.5 build 2172 - CRITICAL security vulnerability fix
This build contains a fix to a CRITICAL SQL Injection vulnerability, as well as fixes to minor security vulnerabilities.

Yesterday (2023-01-14): Finding a minor Path Traversal security vulnerability lead to a throughout process of labeling all untrusted data in the codebase, and during that process a critical SQL Injection vulnerability was also discovered. When exploited successfully SQL Injection allows a malicious actor to read any data from the database and change or delete data. This may expose user salted+hashed Plan web user passwords or other data in the database.

It is recommended to update as soon as possible, even though exploits for the vulnerability may not yet exist in the wild.

The fix has been backported to build 1722 https://github.com/plan-player-analytics/Plan/releases/tag/5.4.1722.1 for those that need it.

This is a first time a this high priority vulnerability affects Plan, so I'm a bit overwhelmed, but I'm hoping to address this vulnerability professionally by releasing a fix in a timely manner, and keeping exact details undisclosed for now to give users time to update.
Change Log
Fixed CRITICAL SQL Injection vulnerability
Details
Vulnerable versions: 5.2 build 1168 to 5.5 build 2163
if login is enabled: Malicious users with permission level 1 (plan.player.other) or 0 (plan.server) can access an endpoint which was found to contain an SQL Injection vulnerability.
if login is not enabled: Any malicious actor can access an endpoint which was found to contain an SQL Injection vulnerability.
Mitigation if you are unable to update
  1. Enable https and login so that less users have access to the vulnerable endpoint.
    https://github.com/plan-player-analytics/Plan/wiki/SSL-Certificate-(HTTPS)-Set-Up
  2. Enable IP Whitelist so that less users have access to the vulnerable endpoint.
Webserver:
Security:
IP_whitelist:
Enabled: true

  1. if unable to update or secure the server, disable Plan Webserver.
    This option is good if you want to delay updating to a more convenient
    time.
Webserver:
Disable_webserver: true
Other fixed security vulnerabilities
  • [Minor] Fixed Path Traversal vulnerability where attacker could gain read access to .css, .js, .png, .woff, .woff2, .eot, .tff files anywhere on the host machine if Customized_files.Enable_web_dev_mode setting was set as true
  • [Minor] Fixed XSS (Cross site scripting) vulnerability in Whitelist deny 403 -page when attacker routes traffic to Plan through a reverse-proxy with malicious X-Forwarded-For header
  • Removed untrusted data from exception messages used within the plugin
    • [Minor] Prevented potential XSS vulnerabilities in Not Found page when untrusted data could enter the error message
    • [Minor] Prevented potential XSS vulnerabilities in Internal Server Error page when untrusted data could enter the error message
  • [Minor] Prevented malicious Hello-packet from breaking Session serialization to CSV on server disable if join address had a ; character in it
Locale
  • Updated Finnish (FI) Locale
5.5 build 2163
This update contains performance improvements and subdirectory support for the new frontend.
Change log
Database
  • Fixed concurrency bottleneck where write and read operations interfered with each other, limiting to one query or transaction from executing at the same time. This bottleneck occurred since the access-lock designed to prevent database operations during schema modifications was still enabled after the schema modifications already completed.
  • Disabled BadAFKThresholdValuePatch - This patch was written to fix bad data input from version 4.5.2 which is no longer being used according to metrics, so this patch can be disabled. It was sometimes executed if a player joined a server and never moved.
Frontend BETA
Export features are now complete, up next is Html Customization.
  • Implemented support and tests for reverse-proxy setups with subdirectory proxy_pass settings (Eg. address.com/plan/)
  • Implemented support and tests for Export to a subdirectory (eg. /var/public_html/plan/ accessed from address.com/plan/)
Webserver
  • Added a read-write lock to json_cache so that files are not read while being written. This might solve some randomly occurring issues.
Plugin Enable
  • Incorrectly written lines in unsaved-sessions.csv during plugin disable are now ignored - a warning is printed instead of an exception stacktrace.
5.5 build 2150 - Hotfix
This update contains a hotfix to build 2144. New installations after build 2100 are not affected - The bug affected instances that were updated from versions prior to build 2100 on networks or fabric servers. Sorry for any inconvenience it has caused. More about the bug below.
Fixed bug in BadJoinAddressCorrectionPatch
An unfortunate typo in session to join address id correction code caused all sessions to get invalid join address id. Instead of correcting invalid ids to correct ids, it changed correct ids to incorrect ids. Any installations where the broken patch ran lost their join address data.
Symptoms of the bad patch:
  • Playtime data too low or missing on player pages
  • Activity index differs between player page and player list
  • Join address data shows no data
Fixes in this update:
  • Fixed the typo, now the patch works as intended and corrects join address ids.
  • Added a second patch that attempts to recover at least some of the missing data by using latest join address in plan_user_info table for installations that ran the bad patch. This is a best-effort solution since the original data was deleted by the bad patch, so some granularity like player changing the address they have used in the past was lost.
  • Playtime and activity index values should recover since the issue was caused by join address ids pointing to invalid numbers.
Top