- Author AuroraLS3
- Creation date
- Featured
Frontend BETA and Database optimizations - Change log 5.4 build 1628
Hello! This update brings some great database optimizations, beginning of Frontend BETA (Opt-in), as well as small fixes. Read more below
Special thanks to Kopo, Pingger & KasperiP for contributions to this update.This update is backwards incompatible due to database schema changes.
Servers with old Plan versions should not be enabled after starting a server with this update installed.
How to prepare for this update
Networks: Make sure to update all servers that are connected to the database!
If an old version starts with the new database schema it will attempt to reverse some of the new changes.
Change log
Database optimizations (Schema changes)
A lot of work was put into analyzing bottlenecks in queries and slow query logs sent in by users.
Benchmarks using 17k players are promising, as /players page now loads in 5s vs 2+min before.
Database patches will apply on first enable of the update to modify the database schema. This can take up to 4 minutes. Networks: Only enable one Plan server and wait for it to patch database before starting other servers.
Changed database schema to optimize query speed and table size
If you're using these columns via Query API you can replace your use case:
If you need uuid in ResultSet, use SELECT u.uuid ... JOIN plan_users u on u.id=<table>.user_id
If you have uuid or server uuid in WHERE, use server_id=(SELECT id FROM plan_servers WHERE uuid='<server uuid>')
Some queries that were identified as slow were optimized to perform better by changing the query
Geolocations queries
Ping country table query
Player table query
Other queries are also sped up by the uuid -> user_id and server_uuid -> server_id changes, as comparing integers is faster than Strings.
Extension data insert was made faster by keeping icon ids in memory for short duration
MySQL insert performance may be improved since auto-commit was turned off
Query page queries now use user ids instead of uuids - this fixes 'Max SQL size exceeded' issues.
Added a new index to plan_ping table to speed up ping queries.
Removed redundant uuid indexes that are now replaced with foreign key indexes for user_id and server_id, which reduces database size.
Frontend BETA (Opt-in)
Frontend is being rewritten in React one page at a time. This is to have more control over what is displayed on the website, since a lot of requested features will require more interaction or hiding features partially. - While doable with vanilla javascript, code would quickly become unmaintainable.
The goal is to get feature parity with the current frontend implementation by rewriting one page at a time and then switch over completely.
Some features will not use React frontend until everything is complete.
To Opt-in set Plugin.Frontend_BETA to true
Player page has been rewritten in React, other pages still use previous implementation.
Html Customization is currently incompatible with the beta and is disabled automatically.
Export is currently incompatible with the beta and old pages continue to be exported even if beta is enabled.
Automated browser testing was improved a lot and all pages are now tested for basic javascript errors.
Data gathering
Fixed AFK time not being gathered on Fabric, the movement detector was incorrectly firing every time a player position packet arrived (Happens once per second).
Config
Default Player head image address was changed from cravatar to crafatar. Remove the whole row from the config and reload Plan if you wish to use the new setting.
Locale
Kopo did a massive locale system improvement in order to support translations on the frontend side, implementing yml conversion, namespacing all of the existing locale keys and writing json endpoints for frontend to load locale - as well as the code responsible for loading those using i18next.
Locale files now use .yml format. Any locale files you may be using are converted automatically.
The new React Frontend allows changing language on the frontend on a per-user basis with a dropdown menu - This is currently only available on the rewritten pages (player page)
KasperiP fixed a typo in Finnish locale
Export
Players page is now exported more frequently
Exported pages should no longer show the Logout button
Extensions
Essentials Economy was split from Essentials Extension. Now there are 2 extensions managed separately. This was done since the economy calls were triggering updates too frequently. If you have disabled Essentials Extension you may need to disable EssentialsEco Extension separately
Fixed NPE related to LuckPerms Extension
Fixed SQL syntax error related to LuckPerms Extension
Fixed issues with KingdomsX Extension when using newer version of KingdomsX (They changed names of some of their packages)
Pingger added support for 1.18 stone blocks to LogBlock Extension
QoL changes
/plan servers command now lists Plan version on each server so it is easier to keep track of servers being out of date.
Bugfixes
Fixed negative register dates in player dat files breaking the database
Fixed ConcurrentModificationException related to AFK tracker
Prevented use of incorrectly formatted WorldAlias regex settings
Fixed Plan failing to enable if database query took too long during enable.
![[IMG]](https://user-images.githubusercontent.com/24460436/163558797-657da8e7-4610-4a5a-96dd-1b5c1b724c42.png "[IMG]")
![[IMG]](https://user-images.githubusercontent.com/24460436/162008595-32ff77f1-d60a-4b6c-881c-e59637582e9d.png "[IMG]")
5.4 build 1516 - Year-in-review
Hello! So, today marks 5 years since Plan development started. That's a pretty long time to be honest. As has become customary, there's a survey and a year-in-review post below this updates change log. Thanks for being awesome!
Special thanks to zax71, KairyByte & OLEGSHA for contributions to this update!
Change log
Survey
- Updated HighCharts to newer version since they have fixed performance issue, in addition I removed ordinal: false workaround in Plan that was used to avoid the performance issue. If you liked the odd gaps in your data that may have been present due to the workaround, you can set Display_options.Graphs.Show_gaps_in_data to true in the Plan config to have them visible.
- Fixed NPE related to null gamemode on some platforms
- Kairubyte added version number to javascript file loading addresses so that they are no longer cached by cloudflare between updates. (Which caused issues in the past)
- zax71 and OLEGSHA added some missing plugin dependencies to softdepend in plugin.yml
Here's a link to this year's survey: https://forms.gle/cpc6trn6wfJCpg8E9
Read on below for a look back on what has been going on with Plan this year & what's in store in the future.
Year-in-review: Year 5
This year I started working as a full-time developer so the updates have been less frequent. Non-the-less, surprising to me, the development speed has not really slowed that much. Thanks to the past years of structuring the code, it is still easy to maintain and add new features to Plan.
Here are the "main" features that landed in the last year.
In terms of features I'd say this year has been very good compared to previous years. I can't claim all the credit, since a lot of this has been made possible by the numerous contributors!
- The Query page
- Player join address gathering
- 15+ new Extensions
- Fabric support
- Network performance tab
- Server uptime
- Database driver downloading
I decided to list all the contributors from 2021 (in order of appearance) here, thank you to: Guinness_Akihiko, galexrt, QuakyCZ, MrFriggo, vacoup, Kopo, Creeperface01, WolverStones, hallo1142, Aurelien30000, BruilsiozPro, WolverStones, BrainStone, AppleMacOS, 4drian3d, EyuphanMandiraci, 10935336, 洛伊, Sander0542, portlek, rymiel, mbax, untuned, PerchunPak, Shadowhackercz, HexedHero, Vankka, DrexHD, zisunny104, SkipM4, MAXOUXAX, adhg6 and BratishkaErik, zax71, KairyByte and OLEGSHA.
This year in numbers
Last year around this time I thought there was a storm brewing in terms of maintainability, but this year has been rather smooth sailing - probably thanks to the reduced update frequency allowing longer testing times for dev builds catching more bugs before they end up in a main release.
- From build 1062 to 1515: 457 commits released as updates
- From #8 to #11 in top spigot plugins by rating: The reduced update frequency has likely impacted relative score compared to other plugins
- From 2500 servers to over 4000 servers
- From 49k lines to 59k lines of code (not counting extensions or web files)
- From 43,7% test coverage to 51.3% coverage
Even though a bunch of tickets get closed all the time (260 tickets closed this year), they never run out. Luckily the new project tracker has turned out to be successful in reducing the ticket mass to manageable chunks that can be worked on whenever I have the extra energy to do so.
To turn towards the future, and adding a slight personal note, I have found a functional balance where things progress, but in bit smaller chunks than before. That said, I don't have nearly as much time for Plan as I used to before I got a job (25h/week vs ~3h/week now), so larger features take quite a long time. In an ideal world I would love to work on Plan full-time, but since Plan covers 1.5% of my monthly living expenses, it is simply not feasible.
Right now I'm also writing a Master's thesis, so Plan development will likely stay on the a backseat at least until I have finished that endeavor. Don't worry though - as you can see from the releases this year, being on the backseat has not really slowed things down
Thanks for reading, and for using Plan! I hope you have a wonderful Christmas and Happy new year!
5.4 build 1489 - feature patch "Further"
Hi! This update has been in the making for quite a while, but it's here now. It has a bunch of new exciting features as well as, maybe exciting bugfixes? There's now a little over 5000 commits! Pretty good pace for ~5 years of development.
This feature patch was codenamed "Further" as it attempts to bring more to the already existing features.
Special thanks to untuned, PerchunPak, Shadowhackercz, HexedHero, Kopo, rymiel, Vankka, DrexHD, zisunny104, SkipM4, MAXOUXAX, adhg6 and BratishkaErik for their contributions to this update!
Change log
Fabric support
- Notice: A warning is now emitted on enable with JVM version < 11. Plan will require Java 11 or newer from version 5.5 onwards.
Kopo, DrexHD, Vankka and rymiel implemented fabric server support for Plan. Big thanks to the ton of work that was put into porting the abstraction layer, listeners and command logic to support fabric servers.
Website
- Fabric is distributed separately as PlanFabric.jar, see Github or Curseforge if you need to install it. Thanks to Kopo for implementing a release pipeline that automatically deploys the fabric version to CurseForge to avoid extra work.
Added Performance tab to Network page
This new performance tab can be used to compare server performance of multiple servers as well as view the performance data gathered by proxy server. You can find it under /network > Servers > Performance
Added server uptime
Network as numbers, Quick view and Server as numbers now show the current uptime of the server. The uptime shows time since last server start. It is calculated from latest plan_tps entry or from time stored in memory at startup.
Query page additions
Other website changes
- Added View selector for servers
You can now select one or multiple servers to view the result data from. This is useful if you want to know players who have played the most on specific servers
- Added has played on server filter to only see players who have joined certain servers.
- Added plugin boolean group filter
You can now filter players by booleans defined by Extensions.
Multiple-select performs players in both (OR-operation). Adding a 2nd 'Has plugin boolean value' -filter allows AND-operation
Added web developer mode
- Added server name to kill lists
- Fixed active session not appearing or affecting numbers on the /player page
- Kopo fixed query page graphs not showing in light mode
- Kopo fixed accordions not opening properly due to missing bootstrap attributes
- Hexedhero changed /player page player head image to include the helm of the skin.
- Kopo updated HighCharts to 9.2.2
- Due to an open bug in HighCharts the line graphs may look slightly odd if you have gaps in data and Display_options.Graphs.Show_gaps_in_data is false - as the gaps will now appear as long lines without points. I think this bug will be fixed some time next week as fix for it is in review
- Kopo added /v1/version (for getting information about current version) and /v1/user (for getting information about logged in user) endpoints preparing for future removal of server side html rendering
Adding new files still needs recompiling the plugin, but this should speed up
- Customized_files.Path can now be used to choose where Html Customized files are placed
(Change this to "<absolute path to git repo>/Plan/common/src/main/resources/assets/plan/web"
for easy time developing)- Customized_files.Enable_web_dev_mode can be used to enable modifications of all files
as well as bypass resource caching for instant updates on browser refresh.
modifications to existing parts of the website considerably.
Html Customization
Config
- Enabled Plan web resources are now exported during Plan enable so you no longer need to visit the page after enabling.
- rymiel implemented a task that checks the modify dates of your custom html files against the ones in the jar and warns if you're using files that are older. (Old files usually cause confusing webpage errors so it's good to have some warning about it on the console.) If you have old files, rename them and reload Plan to generate new versions, then apply your modifications to the new file.
- Note that files generated after modifications made on github are not detected due to lack of data on what version the generated files are. (The last modified date is used)
Locale
- Added World_aliases.Regex-setting that allows defining a regex to use for grouping world names under same alias. (Note that you can also use the existing list to group worlds under the same alias). This feature helps servers where plugins generate large amounts of worlds manage their Plan config - any world that matches a regex won't be added back to the list.
- Incorrect Buycraft secret link was corrected by untuned
- rymiel made it possible to set Display_options.Graphs.TPS.High_threshold and Medium_threshold as decimal numbers (eg 19.5) to allow finer control over these thresholds.
Extensions
- RU Locale was updated by PerchunPak and BratishkaErik
- CS Locale was updated by Shadowhackercz
- CN Locale was updated by SkipM4
- zh_TW Locale was updated by zisunny104
Placeholders
- Vankka made an Extension for LibertyBans
- Fixed issues with Logblock Extension not working (Now it should finally work)
- Fixed issue with newer Quests version and Quests Extension
- MAXOUXAX fixed ViaVersion extension on Velocity & it changed support to 4.x.x versions of ViaVersion.
- Ignored Floodgate related exception
Other
- Added player kill related placeholders
- %plan_player_player_caused_deaths%
- %plan_player_deaths%
- %plan_player_mob_caused_deaths%
- %plan_player_kdr%
- %plan_player_mob_kdr%
- %plan_player_recent_kill_<1-10>%
- %plan_player_recent_death_<1-10>%
- Added %plan_top_{category}_{timespan}_{i}% placeholders
- categories: playtime, active_playtime
- timespans: month, week, day, total
- i: 1 to 10
- Example: %plan_top_playtime_month_1% shows name of the player with highest playtime in last 30 days
- All placeholders that don't start with plan_player_ now support server identifier at the end to show data from another server. Use plan servers command for identifiers. Examples:
- %plan_tps_day:Skyblock%
- %plan_ping_day
roxy%
- %plan_sessions_afk_time_day:Lobby%
- %plan_sessions_afk_time_day:Server 1%
- %plan_tps_day:ed568cd3-f5ab-4893-996b-90fa3b9a9e64% (supports server uuids too)
- Vankka implemented database driver downloading to reduce jar size below Cloudburst and Ore limits
- Fixed NPE on 1.7 servers related to ping method
- Fixed issue with plan db move command when a server had been uninstalled
- Fixed an issue with plan db uninstalled not being able to set Proxy as uninstalled
- Plugin startup should no longer perform a query related to export system on the server thread during enable. (now async)
- Building Plan now requires JDK 16 - NOTE that this does not mean Java 16 is needed to run Plan since it is still compiled to Java 8.
- rymiel fixed Velocity implementation using wrong component serializer for legacy text
- rymiel added a Ore version checker for Sponge to comply with Ore guidelines.
- Fixed NPE when Plan was reloaded at the same time as website was accessed
![[IMG]](https://user-images.githubusercontent.com/24460436/126032064-49103c0e-c6c3-4623-90b3-bfb6e8e92a60.png "[IMG]")
![[IMG]](https://user-images.githubusercontent.com/24460436/126058016-dcb186e6-851c-456f-af23-299df6b0a9f3.png "[IMG]")
![[IMG]](https://user-images.githubusercontent.com/24460436/136654585-278584e4-7e6d-4497-b548-10ae231a02fb.png "[IMG]")
![[IMG]](https://user-images.githubusercontent.com/24460436/136652993-474cd3a6-659b-4238-9bce-d1a04dd023f1.png "[IMG]")
![[IMG]](https://user-images.githubusercontent.com/24460436/126860951-4fff5644-1cdd-44f2-902a-69a7e75209e4.png "[IMG]")