Plan | Player Analytics

Plan | Player Analytics 5.6 build 2883

5.5 build 2150 - Hotfix
This update contains a hotfix to build 2144. New installations after build 2100 are not affected - The bug affected instances that were updated from versions prior to build 2100 on networks or fabric servers. Sorry for any inconvenience it has caused. More about the bug below.
Fixed bug in BadJoinAddressCorrectionPatch
An unfortunate typo in session to join address id correction code caused all sessions to get invalid join address id. Instead of correcting invalid ids to correct ids, it changed correct ids to incorrect ids. Any installations where the broken patch ran lost their join address data.
Symptoms of the bad patch:
  • Playtime data too low or missing on player pages
  • Activity index differs between player page and player list
  • Join address data shows no data
Fixes in this update:
  • Fixed the typo, now the patch works as intended and corrects join address ids.
  • Added a second patch that attempts to recover at least some of the missing data by using latest join address in plan_user_info table for installations that ran the bad patch. This is a best-effort solution since the original data was deleted by the bad patch, so some granularity like player changing the address they have used in the past was lost.
  • Playtime and activity index values should recover since the issue was caused by join address ids pointing to invalid numbers.
5.5 build 2144
This build contains various bugfixes.
Change log
Join Address Data
  • Any join address data that has null characters (gathering was fixed in previous update) is now cleaned on first startup - the data after null character is cleared, and the valid data kept. This should help with join address tab not loading in many cases. This was thoroughly unit tested to ensure it works properly.
Query page
  • Fixed Query results erroring with 500 due to mistake in the code not placing boolean parameters in the query parameters.
Frontend BETA
  • Fixed timezone not being applied to graphs (All line graphs)
  • Fixed exported pages constantly updating data in the background
5.5 build 2121
Another release to add 1.19.3 support to Fabric and release all the goodies developed since previous update :)
Special thanks to DrexHD for contributions to this update.
Change Log
Data gathering
Fabric
  • DrexHD updated fabric code to be compatible with 1.19.3 changes
Frontend BETA
  • Fixed login and register redirects to wrong place
  • Fixed Playerbase overview 30 days and Now being wrong way around on the table
  • React Export is now functional (At least on webservers that serve the bundle at /, further testing needed if your export is on /stats/ for example.)
    • If Frontend beta and export are enabled, the old html files are no longer exported.
    • Extra index.html files are exported to redirect back to the React bundle to allow sharing the URLs to others. This is because in the React frontend the address is handled by React Router and clicking most links on the pages makes no extra requests for html. Reloading on the other hand needs html file, so the extra files redirect to {address}/?redirect={address you used} - so that you end up where you want.
    • Exported pages have a different icon next to last refresh timestamp to visually distinguish from screenshots which one is being used.
    • Known issue with export: The json files are constantly being reloaded by the browser if they're old
5.5 build 2100 - 6 years of Plan!
This release contains a lot of things. Frontend BETA enters next phase since all pages have been rewritten in React, next steps are export and html customization implementation for it. This update also contains data gathering fixes, additions, new placeholders along with other bugfixes.
Today also marks 6th year of Plan development! As is customary at this point there's a Year in Review -post after the change log.
Special thanks to Vankka, angela0930 for contributions to this update
Change log
Data Gathering
  • Removed data after null-byte in Join Address data. Bungeecord and Velocity use the handshake-packet to forward UUID and IP information of players, and since the join address comes from handshake packet, that data ended up in the join addresses Plan gathered. Forge Mod Loader forwards its version information in the same handshake packet in similar fashion. Discarding data after the null-byte should resolve any issues.
Frontend BETA
  • Names of new players are now shown in Light Green in PvP tables when they are the player who was killed within 24h of their first join. This is to help find players who kill newer players.
  • /query page has been rewritten in React.
  • /register page has been rewritten in React
  • Fixed plugin cards overlapping when moving to different page on a table
  • Fixed /network/... redirecting to /network
  • Fixed colors of buttons when hovering mouse on them
I'm excited, the frontend part of rewrite is complete and I can move to implementing export and html customization for the new frontend.
Export
  • Experimental version of export for React version is now enabled if Frontend BETA and Export are both enabled. This will place index.html at the root of the export directory along with all the react files. I have not yet been able to test it properly, so it might be completely unusable. Just throwing it out there.
DataExtension API: Component API by Vankka
If your plugin's data contains chat colors or mini-message information, that information can now be included so that it will eventually show up on the website. Check documentation for the new API here
Extensions
  • Vankka fixed error that occurred when deleting LuckPerms groups
  • Vankka fixed LuckPerms not showing colors properly on the website
Placeholders
Added new placeholders
  • %plan_time_active_{day/week/month}%
  • %plan_time_active_{day/week/month}_raw%
  • %plan_top_player_kills_{day/week/month/total}_{n}%
Locale
  • angela0930 updated ZH_TW locale
Other
  • Fixed ConcurrentModificationException related to world aliases
  • Fixed some queries erroring in Strict group by mode for MySQL
  • Fixed Ore not showing Plan sponge version
  • Fixed removal of cookies that expired while server was offline
  • Fixed RemoveUsernameFromAccessLogPatch applying every time
  • Changed "Database can't keep up with transactions" to less dire message
  • Fixed SQLITE_CONSTRAINT_NOTNULL when request uri is null
Year in review: Year 6
This year has been a lot different from the previous five, since there has been a lot more things going on in my personal life. Last year I got a job, and in January I started writing my Master’s thesis. It was very tough balancing all that on top of developing Plan on the side. I had to take some months off development entirely with a hiatus in the summer and fall to finish up the thesis. In September I returned the thesis for review and I’m happy to now have graduated! Things definitely calmed down after finishing up the thesis and resuming Plan development after the hiatus.

The running theme this year has been steady and stable changes. Inspired by the answers to last year’s poll, the goals for this year were to improve the loading speed of the website, as well as visualizing more of the gathered data.

There’s been a lot of work done this year towards those goals. The database schema was changed drastically, the webserver was changed to Jetty, and Frontend BETA is rewriting the code running in browser to React. Alone each change brings small improvements, but together they improve things a lot.
I’m very excited for the future, since the React frontend speeds up development of new things considerably thanks to environment where changes can be seen instantly. Previously I would have had to compile Plan and restart the test server – It takes 1 second vs 3 minutes it used to take to see changes now.

Main goals for 2023 is to get the new frontend out of Beta after feature parity with previous frontend is reached (Export and Html Customization still to go), and after that it’ll be time to tackle a lot of tickets with ’Visualization’ tag. A lot of the ongoing work has been done to make the future changes easier to do, which should speed up feature additions in the upcoming year.

I’m very happy that all the automation created in the past years is now finally paying off, helping keep confidence in that everything works as intended when changing something. I love that it’s now possible to deliver development builds fast, even within the same hour when someone is asking for help.

Special thanks to Github sponsors, donators, discord moderators, support, and contributors this year! Extra special thanks to Kopo & Vankka for their constant input into the project.

This year in numbers
  • From build 1516 to build 2100: 584 commits released as updates
  • From #43 to #20 in top plugins by rating – and keeping the #8 spot in top spigot plugins by rating.
  • Installations has stayed stable at over 4000 servers
  • From 59k lines of code to 64k (not counting extensions or web code)
  • From 51.3% test coverage to 57.4%
  • 194 issues and 32 (+247 dependabot) pull requests closed this year
Have a good one! :)
5.5 build 1984
This update contains small improvements.
Special thanks to DubHacker for contributions to this update.
Change log
  • [Webserver] Fixed login redirecting back to /login if multiple cookies were present (for example CloudFlare cookies)
  • [Locale] DubHacker updated DE locale
Bugfixes - Change log 5.5 build 1969

Special thanks to PikaMug for contributions to this update!

Change log
Website
  • Fixed login and register pages saying 'You need to specify a password!' incorrectly when using non-default Locale
  • Frontend BETA: Fixed players table on network page linking to wrong URL
Velocity
  • ProxioDev fork of RedisBungee is now supported for Velocity to gather correct network online player count.
Extensions
  • PikaMug updated Quests extension, adding new datapoints and fixing non-repeatable default quests showing as times completed: 0.
  • Fixed Extension tab icons showing wrong icon (Showed the icon for first datapoint)
MySQL
  • Fixed SQL exceptions related to ONLY_FULL_GROUP_BY mode being enabled.
5.5 build 1953 - Hotfixes to build 1951
  • Fixed extensions with only PLAYER_LEAVE not getting updated on player leave
  • Fixed data truncation error related to join addresses on restart when trying to store sessions from last shutdown
  • Self fix: If you're unable to login after updating (usually with reverse-proxy setups), clear cookies.
Change log of previous release, build 1951
5.5 build 1951 - Change log
This release contains update to Java 11, Sponge 8, Fabric 1.19.1, as well as Webserver rewrite to Jetty. In addition it contains new features and improvements, and more progress on Frontent BETA in the form of rewritten server & network pages.

Special thanks to Vankka, Kopo, Drex & stashenko for contributions to this update

Change Log
Breaking changes
This version updates config so backup your config in case you need to revert back to 5.4!
  • Java 11: Plan now requires Java 11 or newer to run. Java 8 is no longer supported.
  • Sponge API 8: Sponge API 7 is no longer supported.
  • Fabric 1.19.1: Fabric 1.18 is no longer supported
If you can't update, you can continue using 5.4 build 1722 even if connected to same database as newer versions. (No database schema changes!)

Supported APIs
  • Updated to Java 11, which was required by Jetty and Sponge API 8.
  • Vankka implemented Sponge API 8 changes.
  • Drex implemented support for Fabric 1.19.1
New features
Jetty Webserver

New webserver implementation speeds up response transmission times considerably
  • Replaced Sun Webserver with Jetty (https://www.eclipse.org/jetty/), an open source embedded webserver library.
  • Plan now supports HTTP/2
  • Plan now supports TLSv1.3 for HTTPS, and browsers have a larger selection of cipher suites they can use.
    • Users of PlanCert.jks (Included in the jar) are encouraged to create their own self signed cert. I had to implement some extra code to support expired certificate (Since PlanCert.jks inside the jar has expired), and that may not be as secure as creating a new self-signed .p12 - Instructions will be added to the wiki, but the plugin also logs the instructions if you use that cert.
  • Fixed gzip compression for responses (speeds up transmission)
Access log
Access logs are now stored in Plan database in plan_access_log table.
  • You can also log access to console with Webserver.Security.Access_log.Print_to_console (default false) in the config. This is useful for debugging reverse-proxy issues.
  • The IP of each request is stored in the database - as it is necessary for security (a valid legal basis for use with GDPR).
  • The request logs in database are removed after Webserver.Security.Access_log.Remove_logs_after_days (default 30)
Swagger
Plan json endpoints are now documented with Swagger available on the webserver itself. You can access the Swagger documentation from /docs on your Plan webserver after updating.

Data Gathering
  • Fabric should now gather join addresses correctly. Previously player IPs were gathered by mistake. To clear the invalid data use /plan db removejoinaddresses {servername} once you install the plugin.
  • Added 'Data_gathering.Preserve_join_address_case' setting (default false) that attempts to preserve join address case. Note that some servers/clients may lowercase addresses regardless.
  • Possibly fixed an issue where Quit event is processed before Join, leading to long incorrect sessions being stored on next join/shutdown
  • Possibly fixed an issue where saving sessions that ended on shutdown failed when another join had just stored one of the join addresses that were previously unseen. (It would retry next reload and probably succeed - so low priority issue.)
Frontend BETA progress
Enable with Plugin.Frontend_BETA setting in the config. Note that Html Customization and Export are not yet supported in the beta.
  • Rewrote /login page in React
  • Rewrote /server page in React
    • You can now switch between visualizations for 'current playerbase' and 'latest join address' pie-graphs
      [IMG]
    • Join addresses by day graph is now available to analyze incoming join addresses further.
      [IMG]
  • Rewrote /network page in React
    • You can now switch between Map projections in Geolocations
      [IMG]
    • You can now sort servers in server list by specific values
      [IMG]
    • Join addresses by day graph also available for network page
    • Added Players online to network performance tab - data from multiple servers can be viewed in one graph on network performance tab
  • Fixed various issues and cleaned up overall appearance
You can track progress of frontend beta here

Bugfixes
  • Fixed lag caused by large tab completion results if server had a lot of players and someone tab completed /plan player .
  • Fixed empty cookies preventing Plan login due to error
  • Reduced maximum join address length to 191 characters due to limitation of old MySQL versions
Configuration
  • Added plan.command permission (default: true) on Spigot servers to allow hiding /plan command from tab completion by negating this permission
  • Config structure was modified to be valid yaml so that other programs can validate it. The config is updated automatically with current values of old config versions.
  • Increased default inactive player removal threshold 180 days -> 3650 days (10 years). This change is not applied automatically to old configs.
  • stashenko updated RU locale
  • Added /plan db migrate_to_online_uuids command to help migrating from an offline-mode server to online-mode
    • Moves data of offline uuids to online uuids (Looks up online uuid from Mojang)
    • Combines data of offline uuids and online uuids if both exist
    • See /plan db for more help
Extensions
  • Plugin method calls should no longer be made during Plan disable.
  • Updated PlotSquared extension, should now register correctly
  • Updated Floodgate extension, should now register correctly
  • Updated Nucleus extension for Sponge 8
  • Removed RedProtect support for Sponge (No longer supported in Sponge 8)
Automation
  • Dev builds for every commit is now available DEV build pipeline (Click on title and scroll down to Artifacts, note that you need to log in to Github.)
Change log 5.4 build 1722
This update fixes fabric 1.19 support and join address gathering.
Special thanks to Kopo for contributions to this update.
Change log
Fix join address gathering (No longer 'Unknown')
An oversight in the way the player leave listener code was structured caused the player join address to be removed from memory before its linking to the session that was ending.
It is now fixed.
Fabric 1.19 support
Kopo fixed fabric pipeline for 1.19 - Unfortunately the changes are incompatible with older versions of fabric. If you need older version of fabric use older version of Plan.
Frontend BETA
  • /players page rewritten in React now available when opted in to frontend beta
    • Note that no fixes to functionality were added to it yet, just implementation in React.
  • Server page rewrite is making good progress and is about 75% complete.
Server identification
  • Possibly fixed ServerInfoFile.yml clearing and server duplicating if MySQL is busy during enable
5.4 build 1690
This update fixes some annoying bugs and prepares the plugin for the move to 5.5 and Java 11 while keeping this version compatible for those unable to update their servers to Java 11.
If no further critical bugs surface next update will be 5.5.
Change log
Plugin stability
  • Fixed issue where Plan would prevent server enable from continuing if database was not responsive during enable.
  • Fixed 2 crashes when database write latency is high
    • Fixed thread starvation crash during high latency when using ViaVersion or ProtocolSupport
    • Reduced chance of out of memory crash when during high latency by starting to drop plugin data transactions for a while if queue grows over 500 transactions. (The transactions continue if queue returns to <50)
Database
Join address schema change
  • Join addresses are now stored in plan_join_address table and id referencing that is added to plan_sessions table for time based analysis implementation later.
    • Backwards compatible schema change - old versions will insert 'unknown' as each sessions address.
    • plan_user_info.join_address column was not removed, but its use is discouraged for 3rd parties as its use may be removed in the future.
Fixes to patches
  • Fixed UserInfoOptimizationPatch resetting join_address data when applied
  • Fixed typo in RemoveDanglingServerDataPatch that caused it to fail
  • Possibly improved speed of RemoveDanglingUserDataPatch
Other fixes to database
  • Prevented transactions from executing if database ran into fatal error. This reduces amount of errors logged on a failed patch
  • Fixed error related to user_id constraint violation for Ping and Session store transactions. If they execute faster than the user register transaction the player will not have up-to-date information until the proper transaction executes, but the storage of other data will no longer fail.
  • Prevented ActiveCookieStore related DbOpException if plugin failed to enable
Website
  • Server page > Performance now lists Average players under Performance as Numbers.
Frontend BETA
  • Implemented player ping graph on the player page when opted into frontend beta
  • Fixed nickname list not showing colors properly
  • Fixed terminal icon not loading when user registered with console
  • Fallback to using relative address if Alternative_IP is misconfigured to reduce confusion. Previously nothing would load if address was different, now there's a chance something loads.
Placeholders
  • Wrote tests for placeholders
    • Fixed player_recent_kill_{1-10} and player_recent_death_{1-10} off-by-one mistake and related error for n=10.
    • Fixed player_unique_players_today erroring if there had been no sessions during today.
  • New placeholders
    • %plan_player_current_session_length%: The length of active session (Online time)
    • %plan_player_current_session_length_raw%: unformatted ms
    • %plan_player_latest_session_length%: The length of active session or previous session if not online
    • %plan_player_latest_session_length_raw%: unformatted ms
    • %plan_player_previous_session_length%: The length of previous non-active session even if online
    • %plan_player_previous_session_length_raw%: unformatted ms
  • player_ placeholders now support adding :playername or :uuid at the end of placeholder to show data for another player
    • Note that if the player by given identifier is not found the placeholder is not replaced.
Other bugfixes
  • Console will now log colors properly on Bukkit and Bungee based servers
  • Fixed some _raw placeholders from being formatted
  • Fixed Error related to GriefDefender Extension
Top