Plan | Player Analytics 5.6 build 2883

Bugfixes - Change log 5.4 build 1633

This update brings a couple important bugfixes to build 1628. Please read change log of 1628 if you're running older build than 1628.

Change log
Database

• Fixed databases failing to patch schema if plan_users or plan_servers were missing rows that had uuids in other tables.
  This affected roughly ~1 in 80 servers based on reports and metrics.
  This problem could occur if server had shutdown at a specific time during player join, or if someone had deleted rows manually from plan_users or plan_servers.
  • The fix deletes dangling data so that the rest of the patches can execute properly.
  • If you have run into this problem on your server this update should fix your database automatically.
• Fixed SQL errors in ViaVersion Extension and ProtocolSupport Extension

Click to expand...

Frontend BETA and Database optimizations - Change log 5.4 build 1628

Hello! This update brings some great database optimizations, beginning of Frontend BETA (Opt-in), as well as small fixes. Read more below

This update is backwards incompatible due to database schema changes.
Servers with old Plan versions should not be enabled after starting a server with this update installed.

How to prepare for this update

Networks: Make sure to update all servers that are connected to the database!
If an old version starts with the new database schema it will attempt to reverse some of the new changes.

Click to expand...

Special thanks to Kopo, Pingger & KasperiP for contributions to this update.

Change log
Database optimizations (Schema changes)
A lot of work was put into analyzing bottlenecks in queries and slow query logs sent in by users.

Benchmarks using 17k players are promising, as /players page now loads in 5s vs 2+min before.

Database patches will apply on first enable of the update to modify the database schema. This can take up to 4 minutes. Networks: Only enable one Plan server and wait for it to patch database before starting other servers.

Changed database schema to optimize query speed and table size

If you're using these columns via Query API you can replace your use case:
If you need uuid in ResultSet, use SELECT u.uuid ... JOIN plan_users u on u.id=<table>.user_id
If you have uuid or server uuid in WHERE, use server_id=(SELECT id FROM plan_servers WHERE uuid='<server uuid>')
Some queries that were identified as slow were optimized to perform better by changing the query
Geolocations queries
Ping country table query
Player table query
Other queries are also sped up by the uuid -> user_id and server_uuid -> server_id changes, as comparing integers is faster than Strings.
Extension data insert was made faster by keeping icon ids in memory for short duration
MySQL insert performance may be improved since auto-commit was turned off
Query page queries now use user ids instead of uuids - this fixes 'Max SQL size exceeded' issues.
Added a new index to plan_ping table to speed up ping queries.
Removed redundant uuid indexes that are now replaced with foreign key indexes for user_id and server_id, which reduces database size.
Frontend BETA (Opt-in)
Frontend is being rewritten in React one page at a time. This is to have more control over what is displayed on the website, since a lot of requested features will require more interaction or hiding features partially. - While doable with vanilla javascript, code would quickly become unmaintainable.

The goal is to get feature parity with the current frontend implementation by rewriting one page at a time and then switch over completely.

Some features will not use React frontend until everything is complete.

To Opt-in set Plugin.Frontend_BETA to true
Player page has been rewritten in React, other pages still use previous implementation.
Html Customization is currently incompatible with the beta and is disabled automatically.
Export is currently incompatible with the beta and old pages continue to be exported even if beta is enabled.
Automated browser testing was improved a lot and all pages are now tested for basic javascript errors.
Data gathering
Fixed AFK time not being gathered on Fabric, the movement detector was incorrectly firing every time a player position packet arrived (Happens once per second).
Config
Default Player head image address was changed from cravatar to crafatar. Remove the whole row from the config and reload Plan if you wish to use the new setting.
Locale
Kopo did a massive locale system improvement in order to support translations on the frontend side, implementing yml conversion, namespacing all of the existing locale keys and writing json endpoints for frontend to load locale - as well as the code responsible for loading those using i18next.

Locale files now use .yml format. Any locale files you may be using are converted automatically.
The new React Frontend allows changing language on the frontend on a per-user basis with a dropdown menu - This is currently only available on the rewritten pages (player page)

KasperiP fixed a typo in Finnish locale
Export
Players page is now exported more frequently
Exported pages should no longer show the Logout button
Extensions
Essentials Economy was split from Essentials Extension. Now there are 2 extensions managed separately. This was done since the economy calls were triggering updates too frequently. If you have disabled Essentials Extension you may need to disable EssentialsEco Extension separately
Fixed NPE related to LuckPerms Extension
Fixed SQL syntax error related to LuckPerms Extension
Fixed issues with KingdomsX Extension when using newer version of KingdomsX (They changed names of some of their packages)
Pingger added support for 1.18 stone blocks to LogBlock Extension
QoL changes
/plan servers command now lists Plan version on each server so it is easier to keep track of servers being out of date.
Bugfixes
Fixed negative register dates in player dat files breaking the database
Fixed ConcurrentModificationException related to AFK tracker
Prevented use of incorrectly formatted WorldAlias regex settings
Fixed Plan failing to enable if database query took too long during enable.

Click to expand...

5.4 build 1583 - Hotfix for MySQL issue
This release contains a hotfix for MySQL driver unloading issues on disable/reload, causing the plugin to spew a really long error message and failing to reload when using MySQL.
Driver is no longer unloaded during reload to avoid any issues.
This bug was introduced in dev build 1536 on January 6th, but it was not caught or reported.

5.4 build 1581 - Bugfixes
This build contains a heck of a lot of bugfixes and a bunch of small features.
Special thanks to HexedHero and Kopo for contributions to this update & thanks to everyone who has been reporting issues!
Change log
Data gathering

• Kopo fixed not logging deaths on fabric when killer was not a player

Config

• Throw more detailed error when Unit of setting is incorrect
• Added Display_options.Player_head_img_url config setting for the player page head image

Stability

• Fixed SpongeForge crashing with Plan on enable. This was caused by a bug in SQLite.
• Possibly fixed OutOfMemory crash where server ran out of threads. Amount of tasks registered by Plan over time was reduced.
• Possibly fixed error on server shutdown related to class loading ("zip file closed")
• Possibly fixed duplicate servers being stored in the database when database is busy during enable.

Website

• Added Join address to /player pages
• Fixed player pages with 2 different UUIDs showing same player's data when both players had same name (In cases where offline and online UUIDs are present due to user error)
• Fixed ISO code for South Korea and North Korea so that they show up properly on the map
• Internal error web pages now have error context
• Added support for Hex color codes in player display names on the website
• Fixed Query page 'to main page' link on reverse-proxy setups using subdirectory
• Fixed performance tab not loading on /network page if locale was not English

Extensions

• Fixed ArithmeticException related to Libertybans extension
• Fixed wrong block count for netherrack in Logblock extension
• Fixed "Connected from same IP" data from Litebans not including player names that had the same IP
• Kopo fixed Litebans extension not getting new data when there were over 5000 rows of specific punishments.
• Disabled Litebans extension automatically on game servers if it is already installed on Proxy to avoid duplicate data
• Clean up data of extension disabled in the config when plugin is reloaded to avoid confusion when a plugin does not disappear from the website after disabling them.
• Luckperms extension now updates its data when player's group is changed / group is deleted even if the player is not online.

Placeholders

• Added new placeholders
  • %plan_server_players_registered_{total/day/week/month}%
  • %plan_network_players_registered_{total/day/week/month}%
  • %plan_server_players_unique_{total/day/week/month}%
  • %plan_network_players_unique_{total/day/week/month}%
  • %plan_player_is_afk%, output true/false
  • %plan_player_is_afk_badge%, output 'AFK'/''
  • %plan_top_{category}_{time}_{n}_value%, output the value giving the player their top spot
• Fixed group by error related to top category placeholder queries
• Placeholders in the Plan config are now refreshed more often and earlier during login to improve chance they are displayed when the other plugin uses server thread for placeholders.

Export

• Hid refresh buttons on exported web files since refreshing does not update data on exported files

Commands

• Fixed fabric specific command issues
  • Fixed /plan reload
  • Fixed commands requiring confirmation expiring the confirmation immediately
  • Fixed console getting wrong kind of messages from command output

Other

• Cleaned up extra error messages from browser dev tools console
• Fixed index out of bounds exception related to trying to log errors after it failed previously (leaving behind empty file)
• Reduced log level of "Webserver disabled (disabled in config)" from warning to info
• HexedHero added more instructions to SQLite exception when temp directory is out of space for when user is using Pterodactyl

5.4 build 1516 - Year-in-review
Hello! So, today marks 5 years since Plan development started. That's a pretty long time to be honest. As has become customary, there's a survey and a year-in-review post below this updates change log. Thanks for being awesome!
Special thanks to zax71, KairyByte & OLEGSHA for contributions to this update!
Change log

• Updated HighCharts to newer version since they have fixed performance issue, in addition I removed ordinal: false workaround in Plan that was used to avoid the performance issue. If you liked the odd gaps in your data that may have been present due to the workaround, you can set Display_options.Graphs.Show_gaps_in_data to true in the Plan config to have them visible.
• Fixed NPE related to null gamemode on some platforms
• Kairubyte added version number to javascript file loading addresses so that they are no longer cached by cloudflare between updates. (Which caused issues in the past)
• zax71 and OLEGSHA added some missing plugin dependencies to softdepend in plugin.yml

Survey
Here's a link to this year's survey: https://forms.gle/cpc6trn6wfJCpg8E9
Read on below for a look back on what has been going on with Plan this year & what's in store in the future.
Year-in-review: Year 5
This year I started working as a full-time developer so the updates have been less frequent. Non-the-less, surprising to me, the development speed has not really slowed that much. Thanks to the past years of structuring the code, it is still easy to maintain and add new features to Plan.
Here are the "main" features that landed in the last year.

• The Query page
• Player join address gathering
• 15+ new Extensions
• Fabric support
• Network performance tab
• Server uptime
• Database driver downloading

In terms of features I'd say this year has been very good compared to previous years. I can't claim all the credit, since a lot of this has been made possible by the numerous contributors!
I decided to list all the contributors from 2021 (in order of appearance) here, thank you to: Guinness_Akihiko, galexrt, QuakyCZ, MrFriggo, vacoup, Kopo, Creeperface01, WolverStones, hallo1142, Aurelien30000, BruilsiozPro, WolverStones, BrainStone, AppleMacOS, 4drian3d, EyuphanMandiraci, 10935336, 洛伊, Sander0542, portlek, rymiel, mbax, untuned, PerchunPak, Shadowhackercz, HexedHero, Vankka, DrexHD, zisunny104, SkipM4, MAXOUXAX, adhg6 and BratishkaErik, zax71, KairyByte and OLEGSHA.

This year in numbers

• From build 1062 to 1515: 457 commits released as updates
• From #8 to #11 in top spigot plugins by rating: The reduced update frequency has likely impacted relative score compared to other plugins
• From 2500 servers to over 4000 servers
• From 49k lines to 59k lines of code (not counting extensions or web files)
• From 43,7% test coverage to 51.3% coverage

Last year around this time I thought there was a storm brewing in terms of maintainability, but this year has been rather smooth sailing - probably thanks to the reduced update frequency allowing longer testing times for dev builds catching more bugs before they end up in a main release.
Even though a bunch of tickets get closed all the time (260 tickets closed this year), they never run out. Luckily the new project tracker has turned out to be successful in reducing the ticket mass to manageable chunks that can be worked on whenever I have the extra energy to do so.
To turn towards the future, and adding a slight personal note, I have found a functional balance where things progress, but in bit smaller chunks than before. That said, I don't have nearly as much time for Plan as I used to before I got a job (25h/week vs ~3h/week now), so larger features take quite a long time. In an ideal world I would love to work on Plan full-time, but since Plan covers 1.5% of my monthly living expenses, it is simply not feasible.
Right now I'm also writing a Master's thesis, so Plan development will likely stay on the a backseat at least until I have finished that endeavor. Don't worry though - as you can see from the releases this year, being on the backseat has not really slowed things down
Thanks for reading, and for using Plan! I hope you have a wonderful Christmas and Happy new year!

5.4 build 1494 - Hotfix to previous build

• Fixed plugins using MySQL that started after Plan from breaking due to Plan unregistering the MySQL driver (Thanks for the fix Vankka!)

There were three things that lead to this issue not being caught:

• The driver unregistering previously dealt with other plugins using Plan driver was forgotten when writing driver download code
• The driver downloading changed so that the mysql driver was not relocated to plan.com.mysql.driver.Driver which caused the main MySQL driver to be unregistered instead
• There was no test environment in which a plugin using MySQL started after Plan

Sorry for any trouble this may have caused.

5.4 build 1490 - Bugfixes to “Further”

• Fixed Ore version checker giving 403 error due to missing User-Agent
• Released version now the same on all release platforms

5.4 build 1489 - feature patch "Further"

Hi! This update has been in the making for quite a while, but it's here now. It has a bunch of new exciting features as well as, maybe exciting bugfixes? There's now a little over 5000 commits! Pretty good pace for ~5 years of development.

This feature patch was codenamed "Further" as it attempts to bring more to the already existing features.

Special thanks to untuned, PerchunPak, Shadowhackercz, HexedHero, Kopo, rymiel, Vankka, DrexHD, zisunny104, SkipM4, MAXOUXAX, adhg6 and BratishkaErik for their contributions to this update!

Change log

• Notice: A warning is now emitted on enable with JVM version < 11. Plan will require Java 11 or newer from version 5.5 onwards.

Fabric support
Kopo, DrexHD, Vankka and rymiel implemented fabric server support for Plan. Big thanks to the ton of work that was put into porting the abstraction layer, listeners and command logic to support fabric servers.

• Fabric is distributed separately as PlanFabric.jar, see Github or Curseforge if you need to install it. Thanks to Kopo for implementing a release pipeline that automatically deploys the fabric version to CurseForge to avoid extra work.

Website
Added Performance tab to Network page


This new performance tab can be used to compare server performance of multiple servers as well as view the performance data gathered by proxy server. You can find it under /network > Servers > Performance

Added server uptime


Network as numbers, Quick view and Server as numbers now show the current uptime of the server. The uptime shows time since last server start. It is calculated from latest plan_tps entry or from time stored in memory at startup.

Query page additions

• Added View selector for servers
  You can now select one or multiple servers to view the result data from. This is useful if you want to know players who have played the most on specific servers
  
• Added has played on server filter to only see players who have joined certain servers.
  
• Added plugin boolean group filter
  You can now filter players by booleans defined by Extensions.
  
  Multiple-select performs players in both (OR-operation). Adding a 2nd 'Has plugin boolean value' -filter allows AND-operation

Other website changes

• Added server name to kill lists
• Fixed active session not appearing or affecting numbers on the /player page
• Kopo fixed query page graphs not showing in light mode
• Kopo fixed accordions not opening properly due to missing bootstrap attributes
• Hexedhero changed /player page player head image to include the helm of the skin.
• Kopo updated HighCharts to 9.2.2
  • Due to an open bug in HighCharts the line graphs may look slightly odd if you have gaps in data and Display_options.Graphs.Show_gaps_in_data is false - as the gaps will now appear as long lines without points. I think this bug will be fixed some time next week as fix for it is in review
• Kopo added /v1/version (for getting information about current version) and /v1/user (for getting information about logged in user) endpoints preparing for future removal of server side html rendering

Added web developer mode

• Customized_files.Path can now be used to choose where Html Customized files are placed
  (Change this to "<absolute path to git repo>/Plan/common/src/main/resources/assets/plan/web"
  for easy time developing)
• Customized_files.Enable_web_dev_mode can be used to enable modifications of all files
  as well as bypass resource caching for instant updates on browser refresh.

Adding new files still needs recompiling the plugin, but this should speed up
modifications to existing parts of the website considerably.

Html Customization

• Enabled Plan web resources are now exported during Plan enable so you no longer need to visit the page after enabling.
• rymiel implemented a task that checks the modify dates of your custom html files against the ones in the jar and warns if you're using files that are older. (Old files usually cause confusing webpage errors so it's good to have some warning about it on the console.) If you have old files, rename them and reload Plan to generate new versions, then apply your modifications to the new file.
  • Note that files generated after modifications made on github are not detected due to lack of data on what version the generated files are. (The last modified date is used)

Config

• Added World_aliases.Regex-setting that allows defining a regex to use for grouping world names under same alias. (Note that you can also use the existing list to group worlds under the same alias). This feature helps servers where plugins generate large amounts of worlds manage their Plan config - any world that matches a regex won't be added back to the list.
• Incorrect Buycraft secret link was corrected by untuned
• rymiel made it possible to set Display_options.Graphs.TPS.High_threshold and Medium_threshold as decimal numbers (eg 19.5) to allow finer control over these thresholds.

Locale

• RU Locale was updated by PerchunPak and BratishkaErik
• CS Locale was updated by Shadowhackercz
• CN Locale was updated by SkipM4
• zh_TW Locale was updated by zisunny104

Extensions

• Vankka made an Extension for LibertyBans
• Fixed issues with Logblock Extension not working (Now it should finally work)
• Fixed issue with newer Quests version and Quests Extension
• MAXOUXAX fixed ViaVersion extension on Velocity & it changed support to 4.x.x versions of ViaVersion.
• Ignored Floodgate related exception

Placeholders

• Added player kill related placeholders
  • %plan_player_player_caused_deaths%
  • %plan_player_deaths%
  • %plan_player_mob_caused_deaths%
  • %plan_player_kdr%
  • %plan_player_mob_kdr%
  • %plan_player_recent_kill_<1-10>%
  • %plan_player_recent_death_<1-10>%
• Added %plan_top_{category}_{timespan}_{i}% placeholders
  • categories: playtime, active_playtime
  • timespans: month, week, day, total
  • i: 1 to 10
  • Example: %plan_top_playtime_month_1% shows name of the player with highest playtime in last 30 days
• All placeholders that don't start with plan_player_ now support server identifier at the end to show data from another server. Use plan servers command for identifiers. Examples:
  • %plan_tps_day:Skyblock%
  • %plan_ping_day
    
    roxy%
  • %plan_sessions_afk_time_day:Lobby%
  • %plan_sessions_afk_time_day:Server 1%
  • %plan_tps_day:ed568cd3-f5ab-4893-996b-90fa3b9a9e64% (supports server uuids too)

Other

• Vankka implemented database driver downloading to reduce jar size below Cloudburst and Ore limits
• Fixed NPE on 1.7 servers related to ping method
• Fixed issue with plan db move command when a server had been uninstalled
• Fixed an issue with plan db uninstalled not being able to set Proxy as uninstalled
• Plugin startup should no longer perform a query related to export system on the server thread during enable. (now async)
• Building Plan now requires JDK 16 - NOTE that this does not mean Java 16 is needed to run Plan since it is still compiled to Java 8.
• rymiel fixed Velocity implementation using wrong component serializer for legacy text
• rymiel added a Ore version checker for Sponge to comply with Ore guidelines.
• Fixed NPE when Plan was reloaded at the same time as website was accessed

5.4 build 1366
This update brings some more bugfixes!

Special thanks to rymiel, Kopo & mbax for contributions to this update
Change log
Website

• Kopo fixed registration modal not opening after pressing register [builds 1354+]
• rymiel fixed Player page plugin tabs not switching with multiple servers having the same plugin
• rymiel changed login and registration to use Request body instead of query parameters - This is more secure as the query parameters could end up in browser history. Previous authentication still works to not break discord bots written by community members, but it is recommended that you move to POST /auth/login with the same previously used query parameters in request body.
• Login page now looks proper (I was getting some uncomfortable feeling about it being slightly different than before)

Extensions

• Kopo fixed ProtectionStones extension causing a chunk load and as result a lag spike (Not a problem in ProtectionStones)
• Fixed Logblock throwing world: null errror
• Fixed Registration of Floodgate extension with Floodgate 2.0 (1.0 no longer supported)
• This was already in previous release, but I forgot to mention it: mbax fixed FactionsUUID extension breaking with newer version of FactionsUUID

Other

• Removed two exception logging calls accidentally left in during debugging
• Started using the new github issue forms
• Moved to a new workflow for implementing issues, separating the tasks of figuring out what needs doing, and doing it.

Change log 5.4 build 1354
This update contains bugfixes and an upgrade to bootstrap 5.

If you got 5 seconds, please answer this one question poll about Html customization https://strawpoll.com/cq5e7vuw6

Special thanks to Kopo for contributions to this update.

Change log

Website

• Kopo updated Plan to use Bootstrap 5, this was a large update, a lot of effort was put to keep appearance of the page the same
• Kopo fixed dark theme not applying to query page table properly
• Kopo removed horizotal scrollbar from players tables so it should look a lot cleaner if you have a lot of plugin data there.
• Some or all Performance graph labels are now hidden depending on the screen width so that the graph is still viewable on mobile.

Bugfixes

• Kopo moved Velocity support to version 3.0.0.
• Fixed session shutdown save exception on spigot 1.17 related to class loading
• Fixed network page breaking due to loadServerPie going missing with some locales
• Fixed ping not being available on 1.17 Spigot
• Fixed AdvancedBan Extension running to NPE when player name doesn't have an UUID

Click to expand...