Changes:EXIT xzm

[dreadbird]

Updated links below

I have Developer buddy 3 will release if anyone wants to try it its the core for NMS5

The general idea is not give too much feedback on the terminal as complexity is bad and with this version im near the limits of bash but it ended with sed and awk so if thats not a clue I dont know else is

asymmetric encryption with a paraphrase
gpg encrypted persistent searches to drawing log gpg stats and stats gpg
if persistent is not wanted can store recent searches in /tmp for hybrid analysis in diff file as well as can draw stats from that
or both

new command recentchanges query will draw stats off the persistent storage
if only using /tmp will draw stats off that
query will also list filter hit count


New tweaks:
Expanded slightly on analytics but not much there is a new option to show total search time
Hybrid analysis in diff file if ANALYTICS on
Option for scrolling terminal look in save-changesnew and recentchanges
Added recentchanges changable options
Modularized so filter is in its own file with no repeats ie having to update compilesearch it will call filter makes the filter easier to manage
compile search sorts the highest profileseds to top

POSTOP system file doctrine can run file info check on search if POSTOP is true

introducing csv or comma seperated values. If you have libreoffice installed its a dream to see the log very nice.
also a command to print the tab seperated doctrine.tsv this is sorted by root and smallest size first useful info ect

As this is the base of NMS5 all comments left in so can tweak it from here to produce the cut and lite versions.
Running the final checks and will post here.


recentchanges query has custom sed and awk commands that can be tweaked to preference

the rsync in porteus is a bit different as it has changes exit so it runs after changes commit if backup option is true.

[dreadbird]

Debugging
This post is only for porteus versions

found bug in porteus version forgot to uncomment line 331 while testing. Needs to be manually uncommented this and auto saving is covered here
https://docs.google.com/document/d/1EJA ... 72sgh069a7

this version will become the porteus version once I test it with 5.1 so if youre on 5.1 stick with version 5 for now as it works with gpg 2

I did it this way because its better to have a key than no key at all if someone has gpg 1.4

developer buddy 5.01 is tested working with porteus 5.1 and gpg 2 as well see instructions for setting up pinentry

This will now be the porteus version as I optimized the functions for decrypting and it does error checking and checks the porteus version everything is all automatic. If something isnt set right it will tell you and not save the gpg file.

as well 5.01 has all the bug fixes and compatibility with changes-commit script just have to set the following:
changes-commit
Make line 104
/opt/porteus-scripts/save-changesnew 6 "guest" "yes" "/mnt/sda1" < --- where /mnt/sda1 is your $BASEDIR from terminal

This is above # Remount aufs with 'udba=none' flag:

[dreadbird]

Debugging
updated links

applied all fixes and tested. ran through emacs flycheck. testing the final release for a month before releasing so not to have 100 bugs to fix. diff file is working as intended and analysis as well. Just going to play around with porteus 5.01 and the older gpg-agent as I think they may be a problem with libreadline.so.6 check into it.

[dreadbird]

https://imgur.com/L7dMKns

Here is the result So now will release this for nemesis and port it to porteus. will release today


Current download links below

instructions
https://docs.google.com/document/d/1EJA ... b4q298tfsg


As python release is working well and before its released I decided to roll all the dev changes into an updated .xzm. Hand checked to ensure most up to date and that you get the best possible product.

save-changesnew Standard changes

Simplified python calling re simpler bash logic
Stabilized database feedback and error handling with attention to readability and future changes
Added dynamic multicore if there are more than 100 files 4 cores will get 16 files or double the amount
Verified database integrity and parsing.
Added cache clear button to clear cache items from db
Made bash scripts more modular and simplified logic
Caught an Rsync glitch where the inc filter was conflicting with the exclude file saving will be half of what it was

I put in better switching for the rnt sym link

rnt or rnt 60 ---> recentchanges n
rnt SRC or rnt 60 SRC or rnt SRC 60
rnt search or rnt search n -----> recentchanges search but inverse and applies filter
rnt search myfile -----> same as filesearch but takes filter off
rnt query

Python database and extremely fast queries. Scalable and simple. Now 45kb for NMS and 52 port

I left it as it is and is very effective as too much data in the database is not good.

Dev changes

Switched from echo to printf to ensure integrity of logfiles
Backend parallel
latest heuristics (Tweaked) for best viewing in /tmp and recursive analysis for statpst
Tweaked checksum with two step verification. Ensures accuracy two key system.
Worked on rsync dynamic exclusions
Looked into regex and modulized save-changesnew. Tested N5 and porteus
Fixed the rsync files vanished error 24. Suppressed and
put a warning re cache item. Trimmed log if anything jams it up.
verified rsync dynamic exclusions and adjust for logfile logic re mode deleted
Overhauled the feedback from Rsync for FEEDBACK option had to parse two ways

Version 5 original features

d2d savings
rollback when on changes mode
moved options for save-changesnew to top
refined hybrid analysis before it was not accurately reflecting ie duplicate entries had to rewrite the ha block especially for checksum
implemented checksum feature for more detailed ha
3 modes. normal safer mode, array mode or multiple core search
When in PSTSTAT analytics is locked out as not to risk security. ie no need to save in tmp if storing in logpst ect
All loops pull with null delimited for speed and accuracy if a file is myfile\noverhere it will capture it.
The search will show a file if there is a file to be found such as a download from pacman ect


I decided to release it as it works and can now port to porteus. The reason I released it now is because its useful to be able to find anyfile if thats what is wanted.
reasonable level of stability as the script got more refined. ie null loops and checks. Thanks for bearing with me as I put this into place and from here hopefully
any tips if someone has or the community has any ideas.

Ignore this part just for release history
5.0 a2 porteus 5.1 5.01 porteus 5.1 5.01 08/20/2025
https://drive.google.com/file/d/1XAAsQ_ ... sp=sharing
53dcfb237dfb9ea5fe96dee629790de08872ae0e8aa74140579a4a33a1929920
Save-changesnew 5.0a1 porteus 5.1 5.01 08/18/25
https://drive.google.com/file/d/1WtCrFJ ... sp=sharing
767d16d16f44d372bfd2f7c7c4565df0f2a66d13257bfb12ba6b2e7567aa357d
sha256
developer buddy 5 nemesis N5 devr2 08/17/25
https://drive.google.com/file/d/1PfCyM_ ... sp=sharing
86d02708a232086543b19d8c8a1b552211fa5b45117929834524f8b87b339160
sha256sum
Save-changesnew 5.0 devr2 08/17/25 porteus 5.1 and 5.01
https://drive.google.com/file/d/1yJnfrZ ... sp=sharing
3f2eb85bc8e9aab70c8aeb500495d145eb80d655f52f11fd9c2db41766bceced
sha256sum
developer buddy 5 porteus rev2a
https://drive.google.com/file/d/15B5Kgq ... sp=sharing
f0bf9d063ab82050987a2d4545c5c689c6a5ec4f sha1sum
heuristics pack . .
https://drive.google.com/file/d/16Z6bY8 ... sp=sharing
a801fc5c45f18ff10fedfd3cd9220714b62e7437
..
developer buddy 5 nemesis N5 rev2a
https://drive.google.com/file/d/1JXHiat ... sp=sharing
89c72f1c2c67e6f70015f47cd44d73fa488bb06a sha1sum
heuristics pack for pulling stats and generating reports with query (includes the one fix regarding if statement)
https://drive.google.com/file/d/19XY4qo ... sp=sharing
f52095fe1b465cbc5a95ada43f093e6990a09d16
Theres one fix to do regarding If statement
developer buddy 5 nemesis N5 rev2
https://drive.google.com/file/d/11C6PuH ... sp=sharing
d1bfec2a6b4bd82db0dbb30166a918201e0ae8b36e49f6aa6ce0c969764dc6cd
sha 256
developer buddy 5 porteus rev2
https://drive.google.com/file/d/1G7WW0K ... sp=sharing
f90eed0b646c4ccf537444af3c72aeeb371c13464a7ae9c4dd60baf46702009a
sha 256
developer buddy 5 nemesis
https://drive.google.com/file/d/1_rhVzV ... sp=sharing
abbcc04ba395ab54ecbbeb2462fa1564dabc6742d536017a1f73dcb28994d7e7 sha256
Early versions
NMS5 https://drive.google.com/file/d/1RVDcGI ... sp=sharing
sha1sum 4045e3b2e88f54c8940bd436dc64b29195b69e2a
porteus 5.1 version 5 https://drive.google.com/file/d/13JDIsz ... sp=sharing
sha1sum 712da2db4e5323ee5d0e4c7c5e58bbb400b715d7
porteus 5.01 version 5.01 https://drive.google.com/file/d/1RbWcZv ... sp=sharing or porteus 5.1
sha1sum bd47f5d047532c60136926528dc226bf8f552b48

[dreadbird]

Alright brought it up to parity so its fully functional with accurate timers ect for pulling.


The query part has to be done by hand you can imagine all the scenarios that could happen if we have 300 log entries we only want to pull the last 10.
So this all has to be done by hand. What I built into it is a report in the query booleans you can generate the report to /Downloads for PST or /tmp for ANALYTICS.

I disabled this by default so its not confusing. Im dialing everything in and I already put the tweak in for both. IE on ANALYTICS I put in pauses so you can see the reports on the terminal. For STATPST no pauses are needed so this makes it somewhat simpler.

The general idea is I have it balanced right now. I cant push too much or things will break. I had to filter out ASCII text and prevent infinite loops in the awk statements.


This lead me to one key refinement. Filter out all files newer than the search range. IE weird system files or cache items that are created in the future. This helps immensly.


So I will post the next update in a month this will give me time to vet and have everything in place. Including Rsync changes I have those already but they must be tested. These are just the updates to the dynamic exclusion building and logfile integrity. What if rsync messes up the log with an error? that has to me trimmed. So ill get that in place. Everything should be working smoothly now


About the update to 2a. The method used to find downloaded files from slackpkg and pacman is ctime > mtime. This works well as it will catch the preserved metadata. But it introduces some inclusions. Luckily I have the section from validprogram that can filter based on range. So what happens is we grab the start time from the mtime of the main search. Then we grab everything equal to or greater than the mtime from the ctime where the downloads were.

Lastly we calculate the range and filter out everything above. We then have a perfect log. So I got that dialed in. I plan to reduce afterwards and that includes this upcoming month. We have to go forward before we can go back. Once we have a stable core and thats all that is needed. So then can reduce and go well below 40kb. As its bash theres no need to go much beyond this. So reduce the query code as well but its useful for proving things first. Then can optimize and as there is already a lot of bash parameter expansion saving a lot of overhead the core is looking good.



Python backend Currently testing will release in a week or less depending on how it goes but its looking very good



I modularized so when recentchanges query is done it looks if youre on database. If not it will use the bash query. If database runs python query and will output the .db to /tmp.

While this is open you can:

Use sqlitebrowser to view the database (logpst , statpst)

I will ship with tkinter which will open a window and show the database is your choose Y y. I managed to get it so you can sort the columns. Its 1mb.



I kept the bash in as well so there will be an option to keep the two gpgs. or use a database and have one gpg. The ha code is much cleaner in python and its scalable. Ill update with the new version next month



The reason behind this app is lets take two situations that happened to me. While making db3 I meant to click properties and I clicked deleted by accident and for some reason that file didnt go to trash. So I couldnt recover the progress I made. Recently in python I was using the official VS code for the python extension. It ate my pstsrg.py when I was complete. Luckily I had an entire backup of my os with save-changesnew.

Also recently I moved nemesis to my ssd as I didnt want to put more wear on my hdd. I took the changes.bak folder and put it directly in the ssd and booted right up with all my changes.

Lastly If I can save someone the time and also you get a search feature to go along with the rsync backup.

[dreadbird]

I updated the ha block with parsing only when necessary

I tested the python version and it worked first time so Ill give it another day because usually thats when you find the worst errors. Then Ill send it out. The speed in python is super fast.

I tweaked it so if theres small changes as the search was being done it will output to the terminal those minor system or cache items. If cdiag is on it will output even more detailed information to the diff file.
It can also detect a hash collision. If in the unlikely event someone edited a file and it has the same hash or if the file somehow has the same hash so two cases. So if you see that message you should buy a lottery ticket lol.

If the file size varies by 3 bytes thats a stealth edit and is caught as well. I made it so its not annoying with a bunch of popups.

With the ha I made it so it only parses when absolutely necessary so as it goes down it will check inode and if its an integer. If not skip the whole block. Then parse the rest of the log.


As there are up to 300 searches the main search loads it in an associative array so it can rapidly grab from the list and avoid grepping all together. For the parallel ha it uses a regular array as only 8 files per process no need to load the entire thing.


heres the new ha block


https://i.imgur.com/ATjcRTI.png

Python block


https://i.imgur.com/GOubEpQ.png

Update

will be releasing today. I put comments at the bottom of both recentchanges and recentchangessearch to automatically open xed or featherpad. This is where if the community has anything they like or dont like I can easily add it. So python backend and its fast.

[dreadbird]

Updated links
Bugging history
changing the name to save-changesnew for both versions as it seems fitting and theres no more changes needed to be done. now fully functional with databases and hybrid analysis. Stripped comments and got it to 45kb and 53kb. Added cache clear button to tk. Expanded on python for readability and maintainability. Verified database and reworked the logic ie I learned its better to make one call to python as bash logic is unwieldy.

I caught a Rsync logic flaw that the inc filter was overriding the exclusion file. So saving takes up 50% less than what it was. I purposely left the rsync code spaced out for maintainability.

Also released on github https://github.com/dreadwrr/Developer-buddy. These are fully updated and independent meaning any updates dont require updates to other scripts.
Changes history
sha256:808487550c1ad61198409d7f8daad279de48740b1781faac289c12b5d90d29ee released database with full ha
save-changesnewNMS Nemesis
sha256:db3c0c5d9219097bea90c652809362b5862443e51c01169fa3c59bff379b1c0f
Quality check
save-changesnew porteus 5.01 5.1
sha256:b9fc3d7b869c45c57f6e754a1718ba14701936b0241a53d729d0c3c3e02662ed ..
sha256:077c8e08d14617bdbb6fe0070304063b13aa600a604c7ba24c04763a86aea481
sha256:886f91b6e6be494249862aefcd5cad5e16be5eca0d2b07e445acdbb1948a69cc
rejected on Quality check
save-changesnewNMS Nemesis
sha256:8abc7a86e56620fb490c9267ff12012bad1fddfe8949960a66ed17bd4212953b
save-changesnew porteus 5.01 5.1
sha256:a14e8b710f4f401c0606eb5f11bf1f6d4d8774ac9e3c5b3d7e5204ecc5176c24
N5 a2 python database / bash
5.0 a2 porteus 5.1 5.01
RecentchangesN5 python database
save-changesnew 5.0a1 .. porteus 5.1 5.01
devr2
5

[dreadbird]

Included in /usr/local/save-changesnew/

Porteus now has rollback and roll summary for non changes=EXIT:/ users which is the same as nemesis graphics mode. A new bootloader entry is made in /syslinux/porteus.cfg with the extramod=. So when you boot up your changes are isolated from changes=EXIT:/ mode.
It will look at the first menu item. If it has changes= it will move that one down so it doesnt conflict. Ie you wouldnt want your .xzm changes to be in your changes drive
Roll back will save the old .xzm as a .bak in /extramod/archive . So in the event of a hiccup or you lose a file you have multiple "Roll backs". If you have ROLLBCK on.

Nemesis already has two menu items graphics and graphics changes so this mimics this.

If you use a grub bootloader make a new entry for Graphics. then point to the line number in grubln="" . and it will auto update. Also a setting below if your bootloader is on a different device.
It will make a backup of grub.cfg everytime it updates it as well. It will only update if the extramod directory is changed so its usually a one time thing.

example

menuentry "Nemesis graphics" {
set root='hd3,gpt4'
linux /boot/syslinux/vmlinuz from=/dev/nvme0n1p4 root=UUID=aeb... <--- this line number it will auto add extramod= even if there is something already there

menuentry "Nemesis25.04 graphics changes" {
set root='hd3,gpt4'
linux /boot/syslinux/vmlinuz from=/dev/nvme0n1p4 root=UUID=aeb00f60-ff73-4318-80ca-c647ea08a619 changes=/dev/nvme0n1p4/porteus
initrd /boot/syslinux/initrd.xz
}


If the variable em is changed it will move all files from this app to the new extramod directory including /archive. If the old extramod directory is empty it removes it from the extramod= cheatcode. So what the logic does is read all items from extramod= and looks in each path. So it will dynamically update the boot loader for convenience and saving you time.

[dreadbird]

save-changesnewNMS Nemesis 09/5/2025
See below link

if your pinentry is acting up see nemesis instructions https://docs.google.com/document/d/1EJA ... v7u9c9ivk3
sha1sum 905b16c459989626c82c9b22dbe70c2c2c91c461

Updated same as the new porteus just porting the parallel python ha for nemesis atm


save-changesnew Porteus 5.01 5.1 09/5/2025
See below link


autobackup=false will make changes-commit non-executable.

if you have your efi bootloader on another drive set bootdvc=""
to use STATPST see pinentry for porteus https://docs.google.com/document/d/1EJA ... zeqt84i55a. tested o.k.
If stuck on gpg1.4 for Porteus 5.01 with gpg agent 2.1.11 see https://docs.google.com/document/d/1EJA ... oka4gzpzc4


manual https://docs.google.com/document/d/1EJA ... it?tab=t.0


tk for viewing the database and sortable columns.
https://drive.google.com/file/d/1IfC8x5 ... sp=sharing

if you are on wayland you may have to run query as actual root but I got it to work with x11 from su - -c. put checks in python to instruct wayland users about the possible display needing to be root.

The following have lists to supress and clear cache. The former is for terminal output supression:
/usr/local/save-changesnew/rntchangesfunctions
/usr/local/save-changesnew/cacheclear
/usr/local/save-changesnew/query.py clear_cache function

see Rsync exclude
https://docs.google.com/document/d/1EJA ... huyjmv98vc

updates:
added custom .svg icon
shellchecked
added autosave option with changes commit tested o.k.
added xzm logic to ensure proper saving. ie if it detects a file changed it will abort and copy to tmp.
updated regex for non changes mode
fixed cacheclear efile to point to /opt/porteus-scripts as opposed to /usr/local/bin (nemesis)
ensures .wh. files are handled properly


Features
If you set d2drive off it will copy the files to /tmp freezing the system. It takes a little longer but ensures a clean backup
If when making an .xzm it detects that a file changed it will stop and recopy to /tmp and insert the .wh. files ensuring a clean .xzm.
.xzm pulling is fast as it copies directly from the changes folder with mksquash
/Downloads save.sfiles.tmp for exactly what files were saved for both the .xzm or save-changesnew

previous hashes
nemesis
1fec1f2ee7411693605af884469cc261501f1899
203d7590011aba4fd68f57d73e4b18c493e722ad
porteus
8b5173679030f82742a795ef04d885188ca47962
9a2509cce8246962b6c590834d340bc9cd373aba

[dreadbird]

I reworked stealth function it can tell if the size is the same but the checksum is different. So web browser I put in an option to suppress common browsers and you can add to a list I have mozilla and .mozilla. I commented out google-chrome another keyword would be chromium-ungoogled for instance. It can also output if a file changed during the search so again system item or cache item. c diag option if on will output to the diff file instead of on the terminal. With a little more information.

If the checksum is different but the modified date is the same that is shown in red on the terminal as well as a hash collision. I built in cache clearing in the script with a list that can be edited and it can clear the ANALYTICS, STATPST and database. The database will call it just to clear the flth.csv. Its all automated using recentchanges query on ANALYTICS if searches are dividable by 5 it tells you about the script. On STATPST if it divides by 50 it notifies about the script. Just clears up junk items. This is important because you want every file to go into the logs. But for stabilizing and metrics you would want to remove the cache items.

A new item Metadata is possible because of the change time in the database. I also included this in ANALYTIC and STATPST default backend. So a lot of information can be gotten and its really fast

casmod will be the downloads as if its y its using change time as modified time. So just tweaking the output. Just making sure inclusions are taken care of but should be able to roll it out shortly

I added hardlinks there is a button to Update hardlinks. A query will be made for files with the same inode and sets hard link count.


archivesrh variable can store rnt.xzm by shotgun saving. So when recentchanges search or rnt search is made it will dynamically look through /tmp and /tmp/rntfiles_MDY*/ folders. from using recentchanges or recentchanges n. So if theres an old search it will grab that and put it in the diff file. If its in range it will show the unique times. If its out of range it will list the entire old search so its useful as it gives you a snapshot of what that time of search was before. Then the hybrid analysis of the new search.

cdiag if enabled will look for collisions
supbrwr will supress cache warnings on terminal from known browsers? Mozilla... ect and also filter items going into the diff file from cdiag.
supress will supress everything on terminal except Critical alerts. Same modified time different checksum and collisions.

I used the supbrwr for both the cer and scr messages. Cer is elevated and Scr is details. with supress on you wont get any alerts about cache items which can be wanted by some users. If Suspect is encountered it stops all other output as thats Critical. Everything is logged in the diff file as well.

A collision isnt critical but would be neat to see if it happens. I customized it so its not boring a possible stealth edit is blue. Warnings are yellow.

Set Hardlinks


I created a hard link and it shows the count. Short of cycling through and actually checking the disk its pretty good. Some files is previous searches may not exist so it will just show the system repeating inodes.


I built in Hard links that are updated as files are inserted. So its a completely accurate count. You can turn this off in the options.

Took me a while to get all the inclusions out. If you go to rntchangesfunctions at the top you can suppress with keywords. This will even filter the Cer alerts in the diff file. It takes all COLLISION and Suspect our before filtering and reinserts them.

You can suppress all terminal output with one of the options. Collisions and Suspect will still show up in red but are really rare so wont be showing.

The most important thing is your system will alert if there is a file modified with the same modified time. Then there are the stealth edits which is also useful

[dreadbird]

bash ha block

https://i.imgur.com/BQ55oOZ.png

python ha

https://i.imgur.com/tOy8Na5.png

https://github.com/dreadwrr/Developer-b ... ag/stndard
source code on github as well as same download links.

Future plans
To get to 40kb even. But stabilized now can maintain easy so will release new versions on GitHub if I discover anything

Thank you to those that participated in the beta.


Update: I have the workings for a new desktop icon just trying to figure out inkscape. I switched over to porteus alpha 2 on usb as the virtual machine wasnt cutting it. I was getting weird slowdowns and also had hyper-v disabled so I dont know what the reason is. might have to try qemu. I also cant stand 60hz so it was very frustrating testing on it. I updated save-changesnew to make sure everything is fine for porteus side of things. Im going to add an option for auto-save for both nemesis and porteus but that will come with a later release.



1.9kb
Played around with inkscape and with the help of the draft from d2jsp forum graphic section I came up with this

For graphics mode and non changes in porteus. The previous _uid_L module is first put in /mtmp. Followed by changes to /mtmp. So this will exclude exclusions first and copy them over. Then delete exclusions from the old one incase exclusions were changed.

there is a setting mdldvc="" that can be set for a different harddrive with d2dmdl="true". If mdldvc is not set it will use $target. conversly it will do it on a different drive.

for a .xzm copy the .xzm directly to the destination. If it detects file changed... it will abort and copy everything to mtmp. That way 99% of the time is way faster and if anything is encountered goes the original way. Then you can put that key word in the exclusion and get 100% faster everytime.

With non linked state each module has the .wh. files when the module limit is reach it will take each _uid_ and merge them into one _uid_ with the changes. If rollback is on it will then take the new _uid_ and archive it. So in that gap between merging any deletion changes done wont appear on boot. But this usually isnt a problem because many instances you dont need to delete files especially if youre just swapping modules and de activating re activating. One thing with _uid_ is if exclusions are changed it only effects that specific module. So if you prefer to have modules remain exactly as they were and combine them manually then non linked state. To apply the deletions turn merging on ect.


On nemesis if you save-changesnew y or yes it will save the backup on changes or graphic changes mode during shutdown reboot. Then set itself to not executable. porteus will just insert the line and if autosave is false is sets changes commit not executable. So both cases its only for changes for nemesis and changes=EXIT for porteus.

[dreadbird]

Summary

_uid_L and _uid_ If you just activate and deactivate modules use non linked state. If you frequently delete files then use linked state. When it merges it will move them to /tmp. incase your memory stick has low memory. If a failure occurs while merging for whatever reason the modules are left in /tmp. A _uid_ module is an exact copy from the system with .wh. and .wh..wh..opq layers. So you may prefer non linked state mode. When the merging is done it will apply the deletions. You might want to set the merge off and do it manually ect. I usually use linked.

Summary for porteus
changes= isolateBACK true

Code: Select all

save-changesnew

a backup will be made
changes=EXIT:/ same but

Code: Select all

save-changesnew backup

or
changes=EXIT:/ isolateBACK false backup true will sync backup to the changes.

If you set backup=false changes commit wont sync the backup also save-changesnew wont sync the backup. if you set autobackup to false and then call the script beforehand it will make changes commit non-executable. so once you have autobackup setup and you dont want it to sync on shutdown just set backup=false is the easiest.

isolateBACK is an interlock if that is set to true the only way to update the backup is to call save-changesnew backup which is how it should be normally the same as nemesis. An interlock is needed because you could switch change modes changes= and changes=EXIT and accidently wreck a backup. and autobackup= sets changes commit to executable so it will automatically save the changes and or backup on shutdown. so this is for if you want a synced backup all the time which some may prefer. even if backup is true if isolateBACK is true it will not sync the backup so theres two levels of protection

confirmed shutdown autobackup with nemesis works. its controlled so on boot it wont be executabled to reset it save-changesnew y or yes. This way if youre doing something important you can call save-changesnew y and it will save the backup on an accidental shutdown or reboot. This proved to be the best method as you dont want the backup constantly syncing. there is no tty so theres no feedback on the terminal but it did save.

[dreadbird]

Example for changes to .xzm



so only the files that changed are listed. This is for both changes=exit and non changes mode.

hi res icon pack https://drive.google.com/file/d/1ipiH5S ... sp=sharing


Putting in parallel python ha as well so the end result of this edition will be faster than before the new option

I finished the parallel python ha. There is a new output regular File that is not normally updated and or is a system file.


Updated links:
viewtopic.php?p=103556#p103556


..
changed save-changesnewfnts line 93 94 to rsync -an and rsync -anv alternatively can use -anc and -anvc

..
..
on nemesis if you cant deactivate the module its because of the startmenu icon or desktop icon. so just delete if you activate deactivate. weird glitch with icons for desktop nemesis

tk for viewing the database and sortable columns.
https://drive.google.com/file/d/1IfC8x5 ... sp=sharing

previous hashes
Nemesis
sha256: b5f30a4a81086e38f3821ca783f179401cac5e3dd07889ecde95be34b74044ca
Porteus
sha256: fc49a7596ff8f4c9ce209f02371a38223b3896644c425a1ed7ff9ea18ef99eac

[dreadbird]

working on mmode="mem" to use python for array handling

While it may have a slight benefit it caused the size to balloon to 62kb. So what I did is put the stat call directly in find in bash which translates to 6 seconds per 1000 new files. So if for some reason there were a lot of new files this would save upwards of 30 seconds ect. And I also put in load balancing.

I commented and spaced out a bit so its clean.

I made sure that any downloaded files will be found and /tmp files are listed in /Downloads. The -SRC tag can be used to package an .xzm and can be called by other scripts ect.



Finalized starting with nemesis. I made a cache file for files over 1mb in /tmp. so on a repeated search it can grab the hash if the stats are same.
xRC is added and tested with pacman it will build the log for the downloads ahead of time. After 1 hour it turns off. These are few files so no overhead.

Parsing captures all filenames with " \ $ spaces and embedded newlines

add entry for scr for metadata what permissions, ownership or group were changed.

Tweaked ha scripts to use parameter expansion in bash and also streamlined all so no unnecessary stat calls.



This shows the behavior of backup or restore. the files have a new checksum but the modified time didnt change. So this output indicates its working and it also automatically supresses other terminal feedback as its critical event. Suspect is for possible malacious tampering and exactly what software would do maintain the original mtime and change it on you

[dreadbird]

Plan to make a python only version. This would use arrays and be probably the same speed but the biggest performance boost would be if there were many new files.

the file size would be smaller in python and after stripping other modes.

Update: the conversion is going smoothly and it turns out you cant convert it very easily so from the start. I put the configs in a .toml file. So recentchanges search is all python and recentchanges is bash for the rnt.xzm and then python.

I got it converted over and the filesize is 49kb. its called by the same script for handling switching. the filter is filter.py and from start to finish all array. So wide compatibility. ill run it through the debugger and see how it goes

As for having it in windows I came across Go and you can write concurrency meaning you can walk the directory tree from multiple directions at once. The results can be saved with sqlite. The initial search would build an index and insert it into a database. then an incremental search can be done fairly quick so folders with a modified date less than the search time skip. system integrity can be verified quickly from the index as well after the initial snapshot is complete. Then use fsnotify to insert new files into the database. the general idea would be to see what files an application uses. So it would work as a find new files feature in windows as well. Python would be used for the gui and then use watchdog or subprocess to Go.

Behind the scenes Go can insert new files into the database while the program is open. If the program closes it can shutoff after an hour. Then when its started again the program can do a refresh on the index. So a quick search can validate integrity very quickly.

Moving forward key improvements:



Was looking at putting the diff file in the main search and then removing before diffing. But the file would add too much detail so as with moving /tmp files out of the main search its best to keep it in a seperate file

The python version is almost complete im designing it as the way forward and for cross compatibility to windows. So its taking longer than I thought but its robust and if done correctly ie the base then it will lead to a successful project. I plan to return shortly to the bash scripts and update things that I noticed and make sure everything is working as intended. It takes a lot of time to maintain both versions so if I can make sure its all correct I dont have to revisit ect.

update: 09/19/2025
have updates for both developer buddy will post today
working on graphic for proteus shield
reviewing uid module saving to ensure its correct.
Added auto -SRC in options. If something compiled to a root folder calling `recentchanges -SRC` can package it by finding the root folder automatically
Minor tweaks identified from making the python version

improvements to _uid_ graphics mode

I came up with an idea to make an .xzm of changes. followed by extracting the last _uid to a different directory. With using rsync -aunc the -n option for a dry run to identify files that changed. Then remove any files from the new .xzm that would be copies from a previous save. For instance if a .xzm is 600 mb from the changes folder and another save is made 5 minutes later the new .xzm could be 5 mb.

a marker file can be placed in /dev/shim TMFS check if its there then its certain its the same session. If rebooted ect the marker file would be gone. Then just make the new .xzm with the cleared changes.

Rolling out the new change for save-changesnew. its experimental so if anything doesnt work just use link state. The new method for non linked state should save a lot of space. So with using graphics mode non changes for the performance boost the _uid_ would be the better option