Random texts

Anything that I feel I have to write down and that I'm not embarassed enough to hide. RSS and ActivityPub (@tokudan@blog.tokudan.de).

mbuffer reads data from an input and writes it to one or more outputs. The more important thing though is the buffering, as you can just tell it to use X amount of memory as a buffer, the default is usually 2 MBytes. Typically the input is stdin and the output stdout, so it works well in a pipe like zfs send | mbuffer | zfs receive but files or TCP connections are possible as well, so it can replace cat and netcat. Tape drives and autoloader have some support as well, but I have no experience with that.

Pipes in shell scripts or directly on the command line are so common that you barely think about them, but they have a serious limitation. They block nearly instantly if the other end of the pipe is not ready to receive the data. This is good and bad at the same time. Good: The sender immediately notices that the receiver has failed, for example. Bad: Unless both ends of the pipe can send and receive data at the same time you lose throughput. Imagine a sender that can average 1 MByte/s output, while using a chunk size of 1 MByte, that means that the sender tries to shove 1 MBytes into the pipe as fast as possible, then has some work to do for nearly a second (e.g. waiting on a hard disk) and then tries to send another 1 MByte chunk. If the receiving end of the pipe does not use the exact same chunk size and e.g. has to do work after each 100 Kilobytes for about a tenth of a second, that means that the receiver is also able to accept about 1 MByte/s, but the reality will look completely different: The sender will generate 1 MByte data and sends the first 100 Kbyte through the pipe, then it locks up, waiting for the reciever for about 0.1 seconds, sending the next 100 Kbyte, waiting again... repeat ten times. In total generating this 1 MByte and sending it through the pipe will take roughly 2 seconds. Meaning that the data rate has just been halved. The generation only takes one second, while the 2nd second has just been wasted trying to shove that data into the pipe.

If you insert mbuffer between those two programs, it looks completely different. While the internal buffer in mbuffer isn't full, it will continue to accept new data from the sender. At the same time, whenever the receiver is ready to accept data, mbuffer will be able to send data from its internal buffer, unless it's empty. With the 2 MByte default buffer size that mbuffer is using, the above example should be running at roughly the full speed of 1 MByte/s, though with a data chunk size of 1 MBytes, I'd probably increase the buffer size to 4 MBytes, just to avoid possible choke points.

The downside is obvious: All that data has to be copied around in memory twice as much, increasing CPU usage. Another downside is that the sending process may think that the receiver got all the data, but actually it's just in mbuffer. This means that mbuffer mainly shines in scenarios where both sides have possible choke points, e.g. because they read and write to relatively slow disks or there's a somewhat unreliable network connection like Wifi in the middle that can choke because of retransmissions.

Another nice bonus: mbuffer displays a running status of the amount of data that went through the pipe:

$ mbuffer < /dev/zero > /dev/null
in @ 9529 MiB/s, out @ 9511 MiB/s, 18.8 GiB total, buffer   1% full ^C
mbuffer: error: error closing input: Bad file descriptor
mbuffer: warning: error during output to <stdout>: canceled
summary: 18.8 GiByte in  1.8sec - average of 10.4 GiB/s

Reading from /dev/zero and sending into /dev/null obviously is only useful as a benchmark, but it works as a nice example.

Homepage: http://www.maier-komor.de/mbuffer.html First version is from 2001 and it is still actively being developed, though the basic featureset has been stable for a long time now. It's available on at least Debian stable and NixOS.

#YetAnotherTool

direnv manages shell environments.

direnv is one of these tools that you basically setup once and after that forget that it's there. You just notice it when it does the job you set it up for and are happy it saves you a lot of hassle.

Enter a directory that you've configured with direnv and it will import things into your environment. That works well for e.g. programming languages where you need specific tools in your PATH or you just need an environment variable to point to a specific file in that environment, like the ANSIBLE_INVENTORY variable. Got two ansible environments in ~/ansible-test and ~/ansible-prod? Drop the following file as .envrc into each one:

ANSIBLE_INVENTORY="$(expand_path hosts)"

You can now cd ~/ansible-test/roles/sshkeys and when running ansible it will use ~/ansible-test/hosts as its inventory file.

Security is good, direnv only executes files that you have authorized by executing direnv allow in that directory. And if the file changes, you need to authorize the file again, so nobody can sneak in bad commands.

direnv also allows importing the environments of other tools like rbenv, Nix, Guix, rvm and node. With the Nix package manager it's even possible to install programs on demand. Add the line use nix -p ansible to the above .envrc and direnv will ensure that ansible is installed when you enter that directory. Leave that directory and ansible is gone again. I'm assuming you don't have it installed system-wide or in your user-profile.

Another way to use direnv comes from @tercean@chaos.social, as he puts it: direnv + git env vars = simple way to manage identities per customer

direnv really helps avoid cluttering your regular shell environment from single-use environment variables and you won't have to remember the names of the files to source to setup a specific environment anymore.

#YetAnotherTool

I kind of wonder why there's no decent user interface for pulseaudio – or if it exists, why it's unknown. Pulseaudio is pretty powerful, but the usability is bad. A simple graph application that lets you connect the dots would go a long way. I remember something like that on Windows about 15 years ago, you could throw in various inputs, outputs and filters and just connect them by dragging lines. Not sure if it still exists or is usable.

Here's what I want to do: 1. Play a game 2. Talk on the microphone 3. Listen to other people on Jitsi

So I have three inputs: Game, Jitsi, Microphone The game sound has to go to both headset and Jitsi recording. The Jitsi output only goes to my headset. The Microphone only goes to the Jitsi recording.

Pulseaudio can create the missing points in betweern very easily. You need two things here: 1. The sink name of your headset output, for me that's alsa_output.usb-Logitech_PRO_X_000000000000-00.analog-stereo. 2. The source name of your headset microphone, for me that's alsa_input.usb-Logitech_PRO_X_000000000000-00.mono-fallback. You can use the following two commands to find them:

# List current sinks:
pactl list short sinks
# List current sources:
pactl list short sources

Then you create the missing points. First the game sink, this will be the output that the game will use:

pactl load-module module-null-sink sink_name=game sink_properties=device.description=game

The you create the sink that Jitsi can record:

pactl load-module module-null-sink sink_name=streamout sink_properties=device.description=streamout

Finally you tell pulseaudio what audio needs to be sent where:

# Loop the microphone into streamout
pactl load-module module-loopback source=alsa_input.usb-Logitech_PRO_X_000000000000-00.mono-fallback sink=streamout
# Loop the game into streamout
pactl load-module module-loopback source=game.monitor sink=streamout
# Loop the game into headset
pactl load-module module-loopback source=game.monitor sink=alsa_output.usb-Logitech_PRO_X_000000000000-00.analog-stereo

Sadly at this point pulseaudio will already start to generate CPU load for copying around and resampling silence...

Now start your game (or any other application) and have it play some sound. Then start pavucontrol. On the first tab named “Playback”, you can find the currently playing applications. There should be a button on the right that lists the current output the game is using. Click it and select “game”. You should now hear it on your headset or whatever output you have decided to use. Next start Jitsi and have it record the “Monitor of streamout”. Finally verify that your microphone is working and you're done.

I've recently started going through my attic again and saw my old #Commodore hardware. I grew up on a #C64 and its 1541 disk drive that was able to transfer about 300 bytes per second into the small 64 kByte RAM that the C64 has. I even got ahold of some more hardware like a C128 and a Plus4 back then, but didn't really do much with them. And I bought a 1581. One of the rare 3½” disk drive that Commodore produced back then, although I was late to the party. I probably bought it around 1994, I believe the german magazine 64'er had a story one day that someone had found a stock of old 1581 disk drives and I absolutely needed one back then. I had no real use for it, but I scraped together my pocket money and got one. I still remember wondering why there was so little data on the 1581 demo disk back then, but never investigated it further. It only had about 10 directory entries. Lacking contact to other 1581 owners I had no way to figure out that my disk was faulty.

Fast-forward to 2020, I just got a connection to a local retro-computing group and got interested in making an inventory of my old hardware and figure out what still works. One person there had modified a PC 3½” disk drive to read all the magnetic flux data that the software or hardware translates into a byte-sequence and wanted to try and make an image of the disk and got me hooked into converting that result into a usable image for an emulator. The image file was an SCP image that could probably be converted by keirf's Disk-Utilities. Awesome programs btw, but it doesn't support d81 images used by the emulators. But the Commodore 1581 apparently uses a common IBM-MFM format that regular PC drives can read, just the two disk sides are swapped and the 1581 uses a completely different logical format: 80 tracks with 40 sectors with 256 bytes each, 254 bytes useable per sector, the first two bytes point to the next track/sector in the chain. So the Disk-Utilities gave me an image that had sectors swapped around. There's a request open to support d81 images, but it's probably low priority, as I provided a small shell script in that issue that can fix the image.

Now here's the interesting part: While looking at the disk in hex editors to figure out what's wrong with it, I saw strings I didn't expect. Strings that looked like a chat log, people talking about copy protection, “...and you can buy a 14,4K baud fax modem for around $100 now...” and similar things.

I had no access to the internet or any BBS during my time on the C64. That text could not originate from me, I didn't even speak english well enough back then.

So I made a mental notice to have a closer look, once I had time and that time has come now.

It seems to be parts of a transcript of a “GeoFAX conference” that was held online by some people, including Maurice Randall where he introduces and answers questions about his new program GeoFAX.

So, here is the text. I've removed Maurice's address and phone number as requested by him.

me one and all, madames and misters to Maurice Randall Live .Chuck> Smoking in the outer lobby only .Chuck> Formal conference rules will apply— .Sam> I'm gonna take my seat & learn something .Chuck> If you wish to ASK A QUESTION PLEASE TYPE ? you will'be recognized in order .Chuck> With no further Ado—-Ladies and Gentleman Maurice Randall!!!! .Chuck> Take it away Maurice .Ellen> ...applause... .Zazatik> clap clap clap .Maurice> Welcome everyone... .John/Cruiser> ...CLAP...CLAP... .Reub> Hip hip!!! .Maurice> And thank you .. I will begin with.. Many of you know that I am working on an 80 column publishing program... Well, it has taken a temporary back seat... On a computer next to me, I have GEOS booted up... And on the screen is an all-new GEOS application.. This new application is called....GeoFax. I believe it is the first program of it's kind for the 64 or 128. We will be able to send and receive faxes with our 64s and 128s from GEOS. Incoming faxes can be saved to disk or printed on your printer or both... Faxes can be saved to disk in two different formats... One is in the form of a GeoPaint file... And the second a GeoFax file... The most common will naturally be GeoPaint. You can also send a GeoPaint file. The conversion to a fax document is handled by the program. This program allows many possibilities other than just faxing. Imagine if you were to connect a real fax machine to your fax/modem with a phone cord... You could use it as a full-page scanner with GeoPaint. The program is a 40 column program and will run on GEOS 64 or GEOS 128 in 40 column mode. The hardware requirements are. A fax modem, a SwiftLink cartridge, and a phone line. And of course your little Commodore 64 Would anybody like to start with a question? .Chuck> Yes John? .John/Cruiser> Assume this is strictly for Geos-Geos transfer? .Maurice> No, it is for GEOS to any fax machine or any other computer running a fax program...This includes any 486 or Mac.. .Chuck> Tom has a question .Maurice> The file that is transferred is a fax document. The GeoPaint is converted for you. .Chuck> Go ahead Tom TOMPATCH> Why not a 80 column? .Maurice> The 80 column version will come after I see what users wish this would do. In the meantime everybody can use this. .Chuck> Yes Doc go ahead. .DOC> what baud rate with the 128? .Maurice> With the 64 and 128, fax transmissions are supported at 2400, 4800, 7200 and 9600. .Chuck> Yes Sam first .Sam> the swiftlink is necessary at 2400? .Maurice> The SwiftLink is' required by the program. This is the fastest way to handle the data and do everything else that it does. .Zazatik> A 128 at 2400 baud would take quite a while to transmit the FAX, correct? .Maurice> Yes, it would be slow, but there are some older modems that only support 2400 baud faxes. .Maurice> It wouldn't really be recommended. end .Chuck> Tom Johnston had a question next. TOMJOHNSTON> it would seem use of an REU/RAMLink is important for obvious reasons of speed..and since a Swiftlink and REU can't be used (easily) together, this also .Maurice> No, the disk routines in GEOS work pretty good. But yes an REU or an HD would be helpful. end TOMJOHNSTON> effectively means you need a RAMLink? .Maurice> Ah, but a SwiftLink and an REU work together easily. .Chuck> Ed you had a statement? .Ed> Just want to interject that modem prices have come way down in recent months, as technology becomes less expensive... and you can buy a 14,4K baud fax modem for around $100 now... Maurice> Yes, so a 9600 or 14.4 modem should be affordable to many people. .Ed> Transmission time also takes into effect how much data is on the page. .Chuck> Robert92 You were next ROBERT92> I am ready to order...Where do I send the money? .Maurice> With GEOS, the REU and SwiftLink plugged into a RamLink is not a problem.Yes, Ed. The program is not supporting the higher resolution (fine mode) for speed reasons. The 80 column version when it is ready will most likely support it though. The program is just getting into it's beta testing... And the manual is being finished..I will be taking orders beginning in the first week of January. Shipping will be in about a month from now. The price will be $39.95 plus $4.00 for shipping. I will also be advertising it in Commodore World and dieHard. Let me tell you a little more about it... On the screen, you see a simulated image of a fax machine... When the fax is coming in, you see the paper scrolling up out of the machine... You get a sort of preview mode this way while it is being saved to disk or printed... There are a series of buttons on the front of the machine where you turn features on or off...Or access additional functions... If you save the incoming faxes as a GeoFax file, you can retain the entire resolution... There are some special printer drivers supplied that allow you to print these files.. The resolution is 216x98 dpi, so as long as you have a printer that supports at least this much, you can print them. Chuck> Sam? .Sam> ok let me see . $100 for fax modem /an REU /Swiftlink /Ram Link / + $45 for pgm ... So how much is that total if I needed it all? ??? not sour grapes just asking.?. .Maurice> You can run this with a minimum system... It only gets better if you have all the good stuff, just like any other application with GEOS. PEACEMAKR.Sam> same ? how much for items ?? .Maurice> Keep in mind that a file or several files can add up in size... So at least a 1581 would be nice. .Chuck> Ellen you had a question? .Ellen> Can they be printed as they come in or do you have to print them from the GeoFax file? In other words... Will it function like a regular FAX machine in that respect?
.Maurice> They can be printed as they come in, instead of saving to disk...Just keep in mind the speed of your printer. It will affect the transmission times. .Chuck> Any further questions? .Chuck> Yes Paul? .Maurice> Did I answer peacemaker's question OK? .Ellen> If you're not home the speed doesn't seem important .Paul/BRAVE1> will a good printer buffer help? .Sam> yes I can look it all up.. .Maurice> A big printer buffer would definitely help. .Paul/BRAVE1> how about a 1750 REU .Maurice> The speed would be important to the person at the other end. Yes? .Ellen> ahhh GOOD point!! .Maurice> Keep this in mind with REU's... The program supports any ram disk that can be used within GEOS, as long as you can also attach a SwiftLink properly. For anybody that has just come in here, this conference is about the new program called 'GeoFax'. .Chuck> Cruiser had a question, then Rich, then Fred .John/Cruiser> Any printer driver that will take advantage of my Epson inkjet? .Maurice> The GeoFax printer drivers will support 240x216, 300x300, and 360x360 dpi printers. .John/Cruiser> Thx .Maurice> Your Epson will be supported. .Chuck> Rich? .Rich> Can the GeoFax switch to disk mode if it is in immediate print mode... .Maurice> If you print them as a GeoPaint, then your standard GEOS printer driver will be used. .Rich> and the printer signals “paper out”?(If used in an unattended way) .Maurice> You will be required to select the modes with the front panel buttons on GeoFax. .Rich> Ok. Thanks. .Maurice> If the printer is a page at a time printer, then the printing will not begin until a page is inserted. Also... This way, an unattended mode will just ignore the calls if a printer or drive problem occurs. .Chuck> Fred you're next. FRED8615> Actually, my question is not about GeoFax. Still want to try it? .Maurice> Go ahead. FRED8615> I'm an amateur astronomer.. I was wondering what you thought the possibility of using Geos... (or not) for CCD cameras. .Maurice> Explain... would it need to interface to the computer? > Yes. .Maurice> All of the port connections on the computer are available and may be used from within GEOS.So it should be possible. .John/Cruiser> Availability target dates for 80-col GeoFax? 80-col Publish? .Maurice> I want to get this one off the ground before I can give an honest answer to that... I've given projection dates on Finally! before and haven't come through on it yet. In about a week, I will upload a demo of GeoFax so that everyone can see what it looks like. .Paul/BRAVE1> will there ever be an upgrade to GeoTerm? If so, will there ever be a Geos front end for delphi? .Maurice> I can tell you this.... With me getting GeoFax going, I am discovering that it is possible to do a fast term program from within GEOS... As far as an upgrade to GeoTerm, no. It would have to be a totally new term program. end .Paul/BRAVE1> CLAP...CLAP...CLAP FRED8615> I joined the conf late, then got thrown off so if this question was asked already I'm sorry... .Maurice> I left a message on the Internet as to how it could be done... If you want I will repeat that here. FRED8615> Does GeoFax send faxes by itself? .Maurice> Do you mean can you program it for certain times? FRED8615> And what version (if any) of Geos is required? .Maurice> GEOS 2.0, 64 or 128, 40 column mode. FRED8615> Can it be programed for time? .Maurice> To send a fax... you begin by clicking on the SEND button... Then select a file from the requester... Then a phonebook pops up... you can select a phone number or type it in manually... or you can select 'MANUAL' and dial out on your handset...'' Then if you are ready... you click on the 'START' button. There are LEDs above the buttons that let you know what is going on... When the START LED goes off, the transmission is finished. FRED8615> Cool. Thank you. .Maurice> This first version won't support programming it for certain times. But it will keep a log file of the transmissions. .Chuck> Yes John? .John/Cruiser> Please “spiral-bind” the docs. .–) .Maurice> Hmmm... that costs a couple bucks a piece. But I have considered it. .Chuck> Ye

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

.Maurice> The program is so easy to use, the docs are almost unnecessary. I like programs that are easy to figure out.
.Chuck> Tom Johnston has a question. TOMJOHNSTON> I was AFK for a while..but... it seems like GeoFax offers a way around GeoGif conversions any thoughts on resolutions?? .Maurice> Well, this is primarily a fax program... But it just happens to be pretty slick for importing stuff. Some day, there might be a program that will allow you to edit a GeoFax document... But for now, you can only print them. .Chuck> Fred had another question FRED8615> Any chance you'll be racing your car at Daytona next year? .Maurice> If I can sell enough fax programs to pay my way down there. The car is ready...But I have to put a motor together... and install the new roof flaps...This all costs money... Something that has been lacking lately. .Chuck> Robert has a question ROBERT92> I just want to say...You will get the money... You may very well have hit the jackpot on this one. FRED8615> I agree. .Maurice> I hope so. It should be a big seller... .ROBERT92> IT will Be!!! .Maurice> I would like to get everyone's opinion on something... I am putting a form of copy protection into the program... but don't get excited! ... Here is how it works... The FCC requires that all faxes be sent with the caller's phone number... So, it must be programmed into the computer and sent with the document... When you order the program...I will put your phone number right into it... This assigns the program to one person... If you buy it from some other source...You give me a call, and I give you a personal id number to type in with your phone number...And it inserts it for you. You can change your number at any time by calling me first. What does everyone think of that id .Chuck> Sam first then Rich then Tom. TOMPATCH> Good idea .Sam> So what happens if you move & change #'s? Never mind you answered it. ROBERT92> Why not? .Maurice> All you do is give me a call and you get a new PIN to enter into the program. .Rich> That was my question too. You're a step ahead of us! Good show! .Maurice> If you sell your computer and your software, the buyer can do the same thing. .Chuck> Go ahead John. .Maurice> If I move? I don't think so, but just in case, I have every registered owners address... .John/Cruiser> I'm nervous about any scheme that relies on the... programmer to be there later on. .Maurice> If something happens to me, there will always be someone around that will take over. This is also why I am asking everyone's opinion on this... I don't like copy protection, but I don't think this way would offend anyone. .John/Cruiser> Ok. Thx. Generally, I just don't like the idea- but you have a right to protect your investment. TOMPAT CH> Repeat, Good idea. .Maurice> It would keep anyone from sharing a copy with someone... because each copy has it's own serial number also. .Sam> It's more for the FCC I think. .Maurice> You will also be allowed to have more than one phone number for $5.00 fee for the second line. FRED8615> Was that the only way you could do it? .Maurice> I don't like other forms of copy protection. Besides it only involves a registration of the software. You don't alter your original, only a copy. But the original can't be used by anyone else, because it is numbered. Unless it gets re-registered by a new owner. .Chuck> John you had another question? .John/Cruiser> I'd still be able to make an archive copy, then? .Maurice> Yes you can make as many copies as you like.Save your original... And you can make extra copies of the installed copy. It can be file copied to any drive you like. .Chuck> Fred you're next, then Tom Patch. FRED8615> Are other devices required, or can GeoFax be used with 1 (or 2) disk drives? .Maurice> It will work with any setup. One drive is ok, just not recommended...One drive is not recommended for anything in GEOS. .Chuck> Tom Patch, then Tom Jonston. TOMPATCH> You would want the area code also? .Maurice> The phone number would include the area code and the rest, You see, if I have your phone number, I can fax stuff to you! TOMJOHNSTON> I've had the same phone # for 12 years, so I suppose I shouldn't complain but...this sounds like a bad idea to me... .John/Cruiser> What about disabling call waiting? .Maurice> You can disable that. It will allow special stuff, even has a simple terminal in it. TOMJOHNSTON> the obvious problem of folks changing numbers seems easily solved..and I trust your good intentions Maurice... but the Commie community has been plagued by “drop-outs”.. and this will likely only be an effective copy protection until some hot dog “hacks” it... .Maurice> I would really rather just trust everyone. I have so far. That's why I am asking opinions. The hac& king part will be tough. Trust me on that one. If I do it, I will do it good. But I can still eliminate it, I'm not sure. .Chuck> Rich then Bob .Rich> As far as copy protections go, this one sounds pretty nice. Plus... whoever buys and registers the s/w... .Maurice> I anticipate always being around.... .Rich> ...would end up handing out his FAX (or voice#) if he thought he could just give someone a copy of it Sounds nice enough. .Maurice> I own a big building in Charlotte MI that I don't intend to move away from..And my wife and I just moved into a house that we plan on staying in now, probably forever... Yes, and the copy can't be registered because the number on it is already registered. The serial number that is. Unless it is legitimately sold to someone else and the original buyer no longer uses it.That is ok. .Chuck> Bob the Tom J again. ROBERT92> I will be sending a fax by 1-30-95? .Maurice> You should be. I will be sending them sooner than that! ROBERT92> I don'T care haw you protect it,it is for me not the county... .Chuck> Tom J and then John again. TOMJOHNSTON> Ok..again I was AFK for awhile. but..will this be available from CMD? .Maurice> Tom has a good point, but I have already thought about me being around for awhile.CMD will be selling it, yes. TOMJOHNSTON> exclusively or will you be distributing yourself. and how many $$$$? .Maurice> Same price. And I will sell it direct also. (I make more money that way) $39.95 plus $4.00 shipping. COD is another $4.00 if desired. I don't take credit cards, sorry. CMD does. TOMJOHNSTON> ..be sure to post your address Maurice.. ];) .Maurice> Maurice Randall P.O. Box XXX XXX XX XXXXX. Street address also. XXXXXXXXXXX. But PO box is all that is needed. PH. XXX-XXX-XXXX If anyone is in the area at any time..Feel free to stop in and say HI. .Chuck> John then Tom P. .John/Cruiser> Never mind... I'm still uncomfortable with copy protection... no easy answer. .Chuck> OK, Tom P? TOMPATCH> When can we send a order and money? .Maurice> Please don't start sending until January... I don't want you to sit on your money any longer than necessary. I've seen too many cases where orders take too long to fill... No fun. .Chuck> Any further questions? TOMJOHNSTON> yeah..I got an off the wall one.. .John/Cruiser> None here. Great job, Maurice. Wish you all the luck in the world on this. CLAP CLAP CLAP .Maurice> Thanks. but Tom, go ahead. TOMJOHNSTON> Anyone ever hear from Dave Ferguson? (DiBief?) .Chuck> I here-by declare the formal part finished! TOMJOHNSTON> yes..thanks Maurice... .Maurice> Your very welcome everyone. .John/Cruiser> HEAR, HEAR! .Reub> Thanks Maurice!! .DOC> nite tom .Maurice> I will do this again, if you like when the program starts shipping. .Ellen> YESSSSSS Please do! Just let me know when Maurice. .John/Cruiser> I would like that very much. .Maurice> And watch for the demo online here. .DOC> oh we are going to get a DEMO version too? .John/Cruiser> ABSOLUTELY! CAN'T WAIT! .Maurice> Chuck has a very simple version of it, but I will upload a better one than he has. ARCA93.Maurice> Thanks for having me here tonight, Ellen. And thank you for the time. .Ellen> It was DEFINITELY our pleasure I hope you'll help out in the FORUM when the program comes out. answering questions about it...ok? .Maurice> I will definitely be supporting it. I will watch the forum for questions on any of my stuff. END OF TRANSCRIPT.

#Hubzilla is a jack-of-all-trades. It's a cloud storage, allows pictures, blogging, allows staying in contact with others. The privacy controls are top-notch. It even has pretty much automated channel migration between hubs if you want to move to another server.

Mastodon probably didn't even notice Hubzilla as it steamrolled past Hubzilla's usage numbers and became the de-facto standard. At least it's also using a public protocol, so Hubzilla can exchange messages. I registered on a private Mastodon server a while back to join that community.

Then I had to share a bunch (>1000) pictures with someone and had a closer look at the cloud storage implementation in Hubzilla. It does the job, but the user experience wasn't that great and it was too complicated for my father. I set up Nextcloud a while back to share files, he's happily using it on his own. Only sent him a username and password and asked him to change the password. Pictures are just files and Nextcloud even allows viewing them on a map.

So it's just some blogging left for Hubzilla. Well, until I saw Writefreely. If I set the Firefox Developer Tools to throttle the download speed to GPRS, my channel page takes more than 60 seconds to load. Writefreely is at about 30 seconds and it responds notably faster. Writefreely's interface is a lot more basic, but it offers RSS and ActivityPub subscriptions, which is more than most blogs do.

The only thing left is reading RSS feeds. That never worked well for me, feeds were only updated every couple of weeks and I never figured out why. As far as I can tell it has something to do with my Hubzilla setup and works fine on other hubs. I've used selfoss for maybe 3 years and have recently switched to FreshRSS, so I never used Hubzilla for that.

I've replaced all my needs with other tools now, so I'll be shutting down my hubzilla server soon.

Thank you Hubzilla and thank you Hubzilla Devs, it's awesome what you managed to do.

New Laptop. Decided to try #KDE/Plasma 5 on it. Battery fully charged. Copied my data. Shut the laptop down to go to an event where I need about 4h of battery life, but as the laptop lasts about 12h (verified this during install) I don't expect any problems. Started the laptop on battery. Logged in. Some search program starts Indexing all files and literally sucks the battery dry to ~60% within just a couple of minutes before I notice and are able to kill the process. Process starts again immediately and continues sucking my battery down to ~35% before I manage to find that stupid switch to really disable it. Why can't these tools check if they're on battery before they launch these damn index processes? Besides that I was never asked if I really want to index all my files (the answer would have been a big NO) and there was absolutely no indicator that such a process was running in the background.

#Baloo

I found myself having to merge two json arrays of objects based on an ID in a shell script. #jq as a tool was pretty much a given and a quick search resulted in several solutions. None of them worked for me, as apparently IDs usually are not numbers anymore and everyone expects them to be strings. My IDs were numbers and jq blamed me for it: jq: error (at <unknown>): Cannot use number (4584) as object key Here is my solution, which is basically copy&paste from JSON joins with minor additions to convert the field into a string.

jq -n --slurpfile file1 groupsize --slurpfile file2 grouplist '
# leftJoin(a1; a2; field) expects a1 and a2 to be arrays of JSON objects
# and that for each of the objects, the field value is a string.
# A left join is performed on "field".
def leftJoin(a1; a2; field):
# hash phase:
(reduce a2[] as $o ({}; . + { ($o | field | tostring): $o } )) as $h2
# join phase:
| reduce a1[] as $o ([]; . + [$h2[$o | field | tostring] + $o ])|.[];

leftJoin( $file2; $file1; .id )
'

nixos-shell is a small shell script written by Jörg Thalheim for Nix and #NixOS. What it does is very simple: It takes a machine configuration from your file, builds a VM with it and runs the VM directly in your terminal with sensible defaults.

$ cat vm.nix
{}

$ nixos-shell vm.nix
/nix/store/6cxb591jsb9n8vwqk9zia1a7l3fji1x6-nixos-vm
Formatting '/home/user/nix/vms/nixos.qcow2', fmt=qcow2 size=536870912 cluster_size=65536 lazy_refcounts=off refcount_bits=16
SeaBIOS (version rel-1.11.2-0-gf9626ccb91-prebuilt.qemu-project.org)



iPXE (http://ipxe.org) 00:03.0 C980 PCI2.10 PnP PMM+1F3910F0+1F2F10F0 C980
Press Ctrl-B to configure iPXE (PCI 00:03.0)...
                                                                              

Booting from ROM...
Probing EDD (edd=off to disable)... k

<<< NixOS Stage 1 >>>

loading module virtio_balloon...
loading module virtio_console...
loading module virtio_rng...
loading module dm_mod...
running udev...
kbd_mode: KDSKBMODE: Inappropriate ioctl for device
starting device mapper and LVM...
mke2fs 1.44.4 (18-Aug-2018)
Creating filesystem with 131072 4k blocks and 32768 inodes
Filesystem UUID: a3b328ea-6db1-410e-bae1-88865281022d
Superblock backups stored on blocks: 
	32768, 98304

Allocating group tables: 0/4done                            
Writing inode tables: 0/4done                            
Creating journal (4096 blocks): done
Writing superblocks and filesystem accounting information: 0/4done

checking /dev/vda...
fsck (busybox 1.29.3)
[fsck.ext4 (1) -- /mnt-root/] fsck.ext4 -a /dev/vda
/dev/vda: clean, 11/32768 files, 6353/131072 blocks
mounting /dev/vda on /...
mounting store on /nix/.ro-store...
mounting tmpfs on /nix/.rw-store...
mounting shared on /tmp/shared...
mounting xchg on /tmp/xchg...
mounting overlay filesystem on /nix/store...
/nix/store/a9i0a06gcs8w9fj9nghsl0b6vvqpzpi4-bash-4.4-p23/bin/bash: line 3: mounts: bad array subscript
mount: mounting /dev/vda on /mnt-root/ failed: Device or resource busy

<<< NixOS Stage 2 >>>

running activation script...
setting up /etc...
Initializing machine ID from random generator.
starting systemd...

Welcome to NixOS 18.09.2030.06808d4a140 (Jellyfish)!

[  OK  ] Created slice system-getty.slice.
[  OK  ] Reached target Remote File Systems.
[  OK  ] Started Forward Password Requests to Wall Directory Watch.
[  OK  ] Listening on udev Control Socket.
[  OK  ] Created slice system-serial\x2dgetty.slice.
[  OK  ] Created slice User and Session Slice.
[  OK  ] Reached target Slices.
[  OK  ] Listening on Journal Socket.
         Mounting POSIX Message Queue File System...
         Mounting Huge Pages File System...
         Starting Remount Root and Kernel File Systems...
         Mounting Kernel Debug File System...
         Starting Create list of required st…ce nodes for the current kernel...
         Starting Load Kernel Modules...
[  OK  ] Listening on Journal Socket (/dev/log).
[  OK  ] Started Dispatch Password Requests to Console Directory Watch.
[  OK  ] Reached target Paths.
[  OK  ] Reached target Swap.
[  OK  ] Listening on udev Kernel Socket.
[  OK  ] Listening on initctl Compatibility Named Pipe.
[  OK  ] Reached target All Network Interfaces (deprecated).
         Starting udev Coldplug all Devices...
[  OK  ] Listening on Journal Audit Socket.
         Starting Journal Service...
[  OK  ] Mounted POSIX Message Queue File System.
[  OK  ] Mounted Huge Pages File System.
[  OK  ] Started Remount Root and Kernel File Systems.
[  OK  ] Mounted Kernel Debug File System.
[  OK  ] Started Create list of required sta…vice nodes for the current kernel.
[  OK  ] Started Load Kernel Modules.
         Starting Apply Kernel Variables...
         Starting Create Static Device Nodes in /dev...
         Starting Update UTMP about System Boot/Shutdown...
         Starting Load/Save Random Seed...
[  OK  ] Started Apply Kernel Variables.
[  OK  ] Started Create Static Device Nodes in /dev.
[  OK  ] Started Load/Save Random Seed.
         Starting udev Kernel Device Manager...
[  OK  ] Reached target Local File Systems (Pre).
[  OK  ] Reached target Local File Systems.
         Starting Rebuild Journal Catalog...
[  OK  ] Started Update UTMP about System Boot/Shutdown.
[  OK  ] Started Rebuild Journal Catalog.
         Starting Update is Completed...
[  OK  ] Started Update is Completed.
[  OK  ] Started udev Kernel Device Manager.
         Starting Networking Setup...
[  OK  ] Started Journal Service.
         Starting Flush Journal to Persistent Storage...
[  OK  ] Started Flush Journal to Persistent Storage.
         Starting Create Volatile Files and Directories...
[  OK  ] Started udev Coldplug all Devices.
         Starting udev Wait for Complete Device Initialization...
[  OK  ] Started Create Volatile Files and Directories.
[  OK  ] Started Networking Setup.
[  OK  ] Started udev Wait for Complete Device Initialization.
[  OK  ] Reached target System Initialization.
[  OK  ] Listening on Nix Daemon Socket.
[  OK  ] Started Daily Cleanup of Temporary Directories.
[  OK  ] Reached target Timers.
[  OK  ] Listening on D-Bus System Message Bus Socket.
[  OK  ] Reached target Sockets.
[  OK  ] Reached target Basic System.
         Starting Name Service Cache Daemon...
         Starting Extra networking commands....
[  OK  ] Started serial-getty@ttyS0.service.
         Starting Kernel Auditing...
         Starting DHCP Client...
[  OK  ] Started Extra networking commands..
[  OK  ] Started Kernel Auditing.
[  OK  ] Started D-Bus System Message Bus.
[  OK  ] Listening on Load/Save RF Kill Switch Status /dev/rfkill Watch.
[  OK  ] Started Name Service Cache Daemon.
[  OK  ] Reached target Host and Network Name Lookups.
[  OK  ] Reached target User and Group Name Lookups.
         Starting Login Service...
[  OK  ] Started Login Service.
         Stopping Name Service Cache Daemon...
[  OK  ] Stopped Name Service Cache Daemon.
         Starting Name Service Cache Daemon...
[  OK  ] Started Name Service Cache Daemon.
[  OK  ] Started DHCP Client.
[  OK  ] Reached target Network.
[  OK  ] Reached target Network is Online.
         Starting Permit User Sessions...
[  OK  ] Started Permit User Sessions.
[  OK  ] Started Getty on tty1.
[  OK  ] Reached target Login Prompts.
[  OK  ] Reached target Multi-User System.



<<< Welcome to NixOS 18.09.2030.06808d4a140 (x86_64) - ttyS0 >>>

Run `nixos-help` for the NixOS manual.
Log in as "root" with an empty password.


nixos login: root

[root@nixos:~/nix/vms]# ls -l
total 26124
-rw-r--r-- 1 root root 26804224 Jan 29 11:58 nixos.qcow2
-rw-r--r-- 1 root root        3 Jan 29 11:57 vm.nix

[root@nixos:~/nix/vms]# 
QEMU: Terminated

I've got several systems to administrate at work. It's common that I have to access them through ssh or copy a file from one system to another. I love the Solaris automounter that's configured on /net by default, meaning that if you access /net/server1/nfsshare2/path/to/file you get exactly what you expect: the file. Now the automounter isn't exactly rocket science and it's easy to setup on a linux system, but I don't want to access everything through NFS and all the security issues that come with that. Luckily, #SSHFS is part of pretty much every linux distribution and the server just requires sftp, which is default on pretty much every system, as it's just a subsystem of the ssh daemon and the ssh daemon handles authentication. So all you need on the server side is an ssh daemon and you need access to login through ssh, preferably through public keys. On the client side all you need is sshfs and the matching ssh client.

$ sshfs myserver:/ /tmp/tmp.FFqyLYuk1X

$ ls -l /tmp/tmp.FFqyLYuk1X/etc/passwd
-rw-r--r-- 1 root root 3679  5. Dez 03:00 /tmp/tmp.FFqyLYuk1X/etc/passwd

This way I can easily access files on the server with local commands on my system.

Having to manually mount every sshfs manually gets tiresome pretty fast. Remember /net on Solaris? The system can easily figure out which server I want to access, so why should I have to mount everything myself?

The automounter is a pretty obvious service that could do that. There's one problem though: automountd runs as root, while the mounts have to run as my own user to be able to access my ssh-agent. I'm sure there are some possible tricks, but hardcoding a root daemon to remotely access the ssh-agent of a user just sounds wrong. Also sshfs isn't a kernel filesystem, it's a filesystem in Userspace (FUSE) and that doesn't seem to work with the kernel automounter. Luckily there's #afuse, that runs as a user and can mount FUSE filesystems.

I would have liked to have this as a systemd user service, but I couldn't figure out how to get sshfs to use my ssh-agent, meaning that all connections would fail. If you have any idea of how to do that... please contact me.

Failing a decent user service managed by systemd, I wrote a simple wrapper, that takes care of running afuse with the necessary options, so my environment.systemPackages in configuration.nix for #NixOS looks like this:

environment.systemPackages = with pkgs; [
	( writeShellScriptBin "afuse-sshfs" ''
		mkdir -p $HOME/sshfs
		exec ${afuse}/bin/afuse -o mount_template='${sshfsFuse}/bin/sshfs %r:/ %m' -o unmount_template='fusermount -u -z %m' $HOME/sshfs
	'' )
] ;

Note: I've discovered that $HOME/sshfs is probably not the best directory for this, you may want to change that e.g. to /sshfs on a single user system or whatever else you fancy.

So now I just have to run afuse-sshfs after login, which I have delegated to the XFCE startup procedure.

The result:

$ ls -l sshfs
total 0

$ ls -l sshfs/myserver/etc/passwd
-rw-r--r-- 1 root root 3679  Dec  5 03:00 sshfs/myserver/etc/passwd

$ df -h | tail -1
myserver:/    125G     11G  108G    9% /tmp/afuse-MHdCcY/myserver

The only issue I have with this solution is that it doesn't seem to automatically unmount the filesystems after some idle time, but as I regularly shut down my system after each work day, that doesn't bother me too much. One more nice feature: Instead of accessing e.g. sshfs/myserver/etc/passwd, I can access sshfs/root@myserver/etc/passwd to force sshfs to login as root. Basically the directory name accepts everything that a simple sftp would accept as well and thus aliases I've added in ~/.ssh/config work just fine.

So... time to finally add some nix content. This one is probably more along the lines of “did you know ... ?”

Just as a heads up, I'll only be using configuration.nix. I don't like the imparative way of installing packages through nix-env.

When I need to add a package on a single system, that's actually very easy, but it depends a bit on how you want to use it. Let's assume that you just want to add it to environment.systemPackages. That's the setting that will take care of adding binaries to /run/current-system/sw/bin (the main binary path on #NixOS) and also results in *.desktop files being parsed as well for desktop environments.

Here I'm adding the package stored in the file ./pkgs/mbuffer/default.nix to the environment. Note that the path is relative to the nix file you put this code in. So if you add it in /etc/nixos/configuration.nix it refers to /etc/nixos/pkgs/mbuffer/default.nix. If you add it to /etc/nixos/imports/mypkgs.nix instead, and you import that file from configuration.nix, it will refer to /etc/nixos/import/pkgs/mbuffer/default.nix.

environment.systemPackages = with pkgs; [
	vlc
	(callPackage ./pkgs/mbuffer/default.nix {})
];

I like to do it this way, because the pkgs/mbuffer/default.nix has the exact same format as all the other package files found in nixpkgs, so if you want to know the exact format of the file, have a look at this file.

This means that you can easily integrate new packages or changed packages into your own system.

Important: It does not work if the package is used in a service, unless that service gives you the option to explicitly overwrite it, for example like services.nginx. You could overwrite the nginx package by specifiying something like the following in your configuration.nix: services.nginx.package = (callPackage ./pkgs/newnginx/default.nix {})