Recent Updates Toggle Comment Threads | Keyboard Shortcuts

  • penguin 10:23 on 2020-05-06 Permalink | Reply
    Tags: ,   

    Homebrew, OpenSSL and PowerShell 

    On Mac, PowerShell really, really wants to specifically use OpenSSL version 1.0.

    Unfortunately, homebrew switched to OpenSSL version 1.1 in v2.2, because OpenSSL 1.0 is end-of-life.

    This fixes it (for now at least):

    brew uninstall openssl --ignore-dependencies
    brew uninstall openssl --ignore-dependencies
    brew install https://github.com/tebelorg/Tump/releases/download/v1.0.0/openssl.rb

    References:

     
  • penguin 12:24 on 2020-04-26 Permalink | Reply
    Tags: , ubuntu, updates   

    Ubuntu update procedure 

    Brain dump.

    # from here: https://askubuntu.com/a/862799/870970
    apt-get update && apt-get --with-new-pkgs upgrade 
    
    # restart
    reboot
    
    # do it again
    apt-get update && apt-get --with-new-pkgs upgrade 
    
    # remove shit
    apt autoremove
    
    
    #
    # UPGRADE
    # 
    
    do-release-upgrade 
    
    
    # done.

     

     
  • penguin 09:49 on 2020-04-26 Permalink | Reply
    Tags:   

    Coding fonts 

    This is a list of popular coding fonts. My personal favorites are marked in bold.

    As for “why” … Consolas and Menlo are just super smooth. Right weight / height / spacing / proportions. You can “just work” with them and it’s never wrong. At the moment I think JetBrains Mono is just this tiny bit better, even if only for the ligatures.

    Honorable mentions go to Iosevka, which is a bit strange, but very narrow so you can fit a lot in the available screen width.

    The rest is not bad, I just keep circling back to one of the mentioned alternatives within minutes usually. Personal style might differ.

    • Consolas – Windows proprietary.
    • Menlo – Mac proprietary.
    • Jetbrains Mono – brew cask: font-jetbrains-mono
    • Fira Code – brew cask: font-fira-code
    • Hack – brew cask: font-hack
    • Monoid – brew cask: font-monoid
    • Iosevka – brew cask: font-iosevka
    • Inconsolata – brew cask: font-inconsolata (broken?!)
    • Source Code Pro – brew cask: font-source-code-pro
     
  • penguin 23:18 on 2020-04-25 Permalink | Reply
    Tags: , , , nextcloud   

    nextcloud and Docker and reverse proxies 

    I have a nextcloud setup like described here (docker-compose, let’s encrypt proxy companion, postgres and nextcloud). And for a while I couldn’t connect any new nextcloud clients to the installation.

    This fixed it:

    <?php
    $CONFIG = array (
      # manually added because it's not picked up from
      # the env vars once set ... it seems ...
    
      # the docker IP range
      'trusted_proxies' => ["172.16.0.0/12"],
    
      # the hostname of the server
      'overwritehost'   => "my.super.secret.server",
    
      # the ENDUSER->PROXY protocol, NOT the proxy-> nextcloud protocol!
      'overwriteprotocol' => "https",
    
      #
      # AAAND NOW back to the original config file ...
      # ...
    )

    Some notes:

     
  • penguin 20:39 on 2020-04-19 Permalink | Reply
    Tags: avm, fritzbox, home network, unifi, usg   

    FritzBox, Unify Security Gateway, and router replacements 

    My respect for network changes went up two or three … hundred notches.

    What happened. At my mother’s place, there was a German FritzBox, which handles a VPN between my place and hers, phone lines, the doorbell, WiFi, DHCP, and maybe even more. Now we rent our parts of our house with AirBnB, and those people want – WiFi.

    I don’t want them to be in the same WiFi as myself and – more important – my mother, also I wanted to manage the whole house-WiFi from one central place. Unify / Ubiquity does a great job of delivering really capable products to a reasonable price, so I went for it. During the whole weekend (really, every waking second except for two 10k runs to get the anger out of my system) this is what I learned:

    • This article about how to configure a FritzBox to just do the modem part is still and fully correct for 1und1 in Germany.
      • FritzBox PPPoE passthrough might not work if the model is too old. It won’t tell you anything.
      • You don’t need to set a VLAN ID in the USG then.
      • You do not need to prefix the 1und1 username with “H”, at least for my contract.
    • You must not configure any duplicate IPs in the Unify Control Center, ever. The thing works declaratively – if you hit “apply” or “save” the Control Center tries to create this scenario.
      • What do I mean? I configured the local “network” in the USG with the existing router/gateway IP. But when hitting “save” the USG is immediately configured to use that IP, battling the FritzBox, and provisioning will fail.
    • A FritzBox cannot connect two site networks without the usage of the proprietary “MyFritz” service out of the box. You have to tinker. Which sucks. (Really, AVM?! No site2site VPNs without MyFritz?!)
      • This is the last open item btw, this is the only thing I couldn’t rebuild so far.
    • “systemd-resolved” really want to use DNSSEC. It shouldn’t when using a USG as name server though.
    • /etc/systemd/resolved.conf does not like comments after settings (“THING=value # with a comment behind” will not work)
    • cron jobs are “out”, using systemd timers is “in”.
    • a systemd timer has to be treated the same way as a systemd service (“systemctl enable thing.timer ; systemd start thing.timer“)
    • systemd really wants to do everything
      • … but I kinda like it, it’s one thinking behind everything, and really flexible
      • … if it works.
      • … which it sometimes really doesn’t
      • … and then you have no f*cking clue why.

    In the end, I’m about 90% there. The results are very, very good – Unifi does a great job in providing ready-for-use products which satisfy SMB requirements just fine. Some annoyances, well, really solid and flexible and insanely useful nonetheless.

     
  • penguin 22:23 on 2020-04-16 Permalink | Reply
    Tags: camel, , , success   

    A more peaceful rant about Java & Camel 

    I did it. I feel like I survived something, like I am on the path on enlightenment (and it feels like raising the foot for the first step in a yourney which I know to be several AEs long). Anyway, I’m kinda happy.

    What did I do? I succeeded to write this Java code using the Apache Camel library (and yes, that’s it – in total):

    import org.apache.camel.main.Main;
    
    public final class CamelPusher {
    
        private CamelPusher() {
        }
    
        public static void main(String[] args) throws Exception {
            Main camelMain = new Main();
            // https://is.gd/P8SzKX
            camelMain.configure().withXmlRoutes("file:*.xml");
            camelMain.run(args);
        }
    
    }
    

    Impresive, huh?

    What took me longest was to add the “file:” in the line camelMain.configure().withXmlRoutes("file:*.xml");. Yup, really.

    Code-wise, that is. It took me even longer to set this up:

    <?xml version="1.0" encoding="UTF-8"?>
    <project xmlns="http://maven.apache.org/POM/4.0.0" 
      xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
      <modelVersion>4.0.0</modelVersion>
      <groupId>de.company.business</groupId>
      <artifactId>camelocclient</artifactId>
      <version>1.0.0-SNAPSHOT</version>
      <name>CamelOCClient</name>
      <description>The thing that transfers payment files to OC</description>
    
      <properties>
        <maven.compiler.source>1.8</maven.compiler.source>
        <maven.compiler.target>1.8</maven.compiler.target>
      </properties>
    
      <repositories>
        <repository>
          <id>spring-releases</id>
          <url>https://repo.spring.io/libs-release</url>
        </repository>
      </repositories>
      <pluginRepositories>
        <pluginRepository>
          <id>spring-releases</id>
          <url>https://repo.spring.io/libs-release</url>
        </pluginRepository>
      </pluginRepositories>
    
      <dependencies>
        <!-- https://mvnrepository.com/artifact/org.springframework/spring-context -->
        <dependency>
          <groupId>org.springframework</groupId>
          <artifactId>spring-context</artifactId>
          <version>5.2.5.RELEASE</version>
        </dependency>
    
        <dependency>
          <groupId>org.apache.camel</groupId>
          <artifactId>camel-core</artifactId>
          <version>3.2.0</version>
        </dependency>
    
        <dependency>
          <groupId>org.apache.camel</groupId>
          <artifactId>camel-ftp</artifactId>
          <version>3.2.0</version>
        </dependency>
    
        <!-- https://mvnrepository.com/artifact/org.apache.camel/camel-spring -->
        <dependency>
          <groupId>org.apache.camel</groupId>
          <artifactId>camel-spring</artifactId>
          <version>3.2.0</version>
        </dependency>
    
        <!-- https://mvnrepository.com/artifact/org.slf4j/slf4j-simple -->
        <dependency>
          <groupId>org.slf4j</groupId>
          <artifactId>slf4j-simple</artifactId>
          <version>1.7.30</version>
        </dependency>
    
      </dependencies>
    
      <build>
        <plugins>
          <plugin>
            <groupId>org.codehaus.mojo</groupId>
            <artifactId>versions-maven-plugin</artifactId>
            <version>2.7</version>
            <configuration>
              <allowMajorUpdates>false</allowMajorUpdates>
              <!-- https://stackoverflow.com/a/22174801/902327 -->
              <!-- fucking maven shit honestly those java assholes suck so badly -->
              <rulesUri>file:///${project.basedir}/pom-versions-ruleset.xml</rulesUri>
            </configuration>
          </plugin>
          <plugin>
            <!-- see here: https://is.gd/HNFI0H -->
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-maven-plugin</artifactId>
            <version>2.1.12.RELEASE</version>
            <executions>
              <execution>
                <goals>
                  <goal>repackage</goal>
                </goals>
              </execution>
            </executions>
          </plugin>
          <plugin>
            <artifactId>exec-maven-plugin</artifactId>
            <version>1.6.0</version>
            <groupId>org.codehaus.mojo</groupId>
            <executions>
              <execution>            <!-- Run our version calculation script -->
                <id>git-tag</id>
                <goals>
                  <goal>exec</goal>
                </goals>
              </execution>
            </executions>
            <configuration>
              <executable>git</executable>
              <arguments>
                <argument>tag</argument>
                <argument>${parsedVersion.majorVersion}.${parsedVersion.minorVersion}.${parsedVersion.incrementalVersion}${parsedVersion.qualifier?}</argument>
              </arguments>
            </configuration>
          </plugin>
        </plugins>
      </build>
    
    </project>
    <!-- I FUCKING HATE MAVEN. -->
    

    And this, cause maybe I still don’t fully get maven (and with maybe I mean definitely, and yes I know there’s an error in the Makefile, I’m gonna fix it – later):

    all: clean build prepare-test test
    .PHONY: all
    
    clean:
        rm -rf tmp
    .PHONY: clean
    
    build:
        mvn package
    .PHONY: build
    
    prepare-test: clean
        mkdir -p tmp/from_here tmp/to_here tmp/and_from_here tmp/to_here_as_well
        echo deeply > tmp/from_here/i-want-to-be-moved.txt
        echo deeply > tmp/and_from_here/hashtag-metoo.txt
        cp target/camelocclient-SNAPSHOT.jar tmp/
        cp test-config.xml tmp/
    .PHONY: test
    
    test:
        cd tmp; pwd; java -cp . -jar camelocclient-SNAPSHOT.jar
    .PHONY: test
    
    run: test
    .PHONY: run
    
    .PHONY: _git_dirtycheck
    _git_dirtycheck:
        [[ ! -z "git status -s" ]] || (echo -e "\n\n***** working dir is dirty. *****\n\n" && false)
    
    .PHONY: _major
    _major: _git_dirtycheck
        mvn build-helper:parse-version \
            versions:set \
            -DnewVersion=\$${parsedVersion.nextMajorVersion}.0.0 \
            versions:commit
    
    .PHONY: _minor
    _minor: _git_dirtycheck
        mvn build-helper:parse-version \
            versions:set \
            -DnewVersion=\$${parsedVersion.majorVersion}.\$${parsedVersion.nextMinorVersion}.0 \
            versions:commit
    
    .PHONY: _patch
    _patch: _git_dirtycheck
        mvn build-helper:parse-version \
            versions:set \
            -DnewVersion=\$${parsedVersion.majorVersion}.\$${parsedVersion.minorVersion}.\$${parsedVersion.nextIncrementalVersion} \
            versions:commit
    
    .PHONY: _set_version
    _set_version: _git_dirtycheck
        read -p "new version: " VERSION ; \
        mvn build-helper:parse-version \
            versions:set \
            -DnewVersion=$$VERSION \
            versions:commit
    
    .PHONY: _snap
    _snap: _git_dirtycheck
        mvn build-helper:parse-version \
            versions:set \
            -DnewVersion=\$${parsedVersion.majorVersion}.\$${parsedVersion.minorVersion}.\$${parsedVersion.nextIncrementalVersion}-SNAPSHOT \
            versions:commit
    
    .PHONY: _tag
    _tag:
        git commit -am "Version bump"
        mvn build-helper:parse-version exec:exec
    
    .PHONY: _snap_tag
    _snap_tag: _snap _tag
    
    .PHONY: _addsnap
    _addsnap:
        mvn build-helper:parse-version \
            versions:set \
            -DnewVersion=\$${parsedVersion.majorVersion}.\$${parsedVersion.minorVersion}.\$${parsedVersion.incrementalVersion}-SNAPSHOT \
            versions:commit
    
    .PHONY: _release
    _release: _git_dirtycheck
        mvn build-helper:parse-version \
            versions:set \
            -DnewVersion=\$${parsedVersion.majorVersion}.\$${parsedVersion.minorVersion}.\$${parsedVersion.incrementalVersion} \
            versions:commit
    
    .PHONY: major
    major: _major _tag build _snap_tag
    
    .PHONY: minor
    minor: _minor _tag build _snap_tag
    
    .PHONY: patch
    patch: _patch _tag build _snap_tag
    
    .PHONY: addsnap
    addsnap: _addsnap
    
    .PHONY: rel
    rel: _release _tag build _snap_tag
    
    .PHONY: release
    release: rel
    
    .PHONY: set-version
    set-version: _set_version _tag build _snap_tag
    
    .PHONY: setver
    setver: set-version
    

    So this is it. That abomination took me days to build.

    So now I can do “make release“, get a .jar file, and place something like this next to it:

    <?xml version="1.0" encoding="UTF-8"?>
    <!--
      DOKUMENTATION:
      https://camel.apache.org/manual/latest/spring.html
    -->
    
    <routes xmlns="http://camel.apache.org/schema/spring">
        <route>
            <from uri="file:from_here"/>
            <to uri="file:to_here"/>
        </route>
        <route>
            <from uri="file:and_from_here"/>
            <to uri="file:to_here_as_well"/>
        </route>
    </routes>
    

    … having a freely configurable, all-purpose, no-nonsense Camel engine to my disposal, which I can then deploy to any host of my liking to do things which are useful. In that case – it’s supposed to transfer files (the “left” side of the system, a couple of deployments) to S3, and then pull them out (the “right” side of it, one deployment) into a folder on a target machine.

    This is it, this is all, this is basically the most trivial use case ever invented, and it’s supposed to be replaced with something “real”. Let’s see how long it takes to get there.

    Now, am I happy? Yes. Is this … “good”? No. Does it work? Probably (it does with a file server / sftp transfer system).

    But the best part is: I kinda, sorta understand a little bit of what I did. And I am starting to think that Camel might actually be awesome, and maven actually useful – once you start understanding the concepts and ecosystem. But that’s just a vague possibility, given that I still don’t 😉 .

    I published the files here as well.

     
  • penguin 17:47 on 2020-04-13 Permalink | Reply
    Tags: ,   

    A big fucking rant about Java 

    Well, I’m back to writing Java. It was supposed to be simple – an Apache Camel .jar file which just reads the route definitions from a nearby *.XML file in the same directory. So it can be re-used, because we have a couple of dead simple use cases which just require a Camel runtime and no special Java Beans.

    Simple, right.

    I’m playing Doom Eternal for fun at the moment. And just thinking of this abomination called “maven”, which Java uses to build shit makes me wish for all maven devs to rot in hell eternally. This piece of junk is basically undocumented. Don’t get me wrong, theres tons of written stuff, intros, tutorials, etc. about maven. It’s just after reading all of them you’re still the same code-copy-pasting monkey you were before. Once you hunted down the snippet on Stackoverflow you go on to the next blackbox.

    Examples? Sure.

    • Versions in build plugins. Maven complains, but countless examples I found don’t use any version. (A Java illness, it seems all examples have to be incomplete, otherwise it’s no fun for the developer, right?).
    • Complexity. You can configure the versions-maven-plugin to update the version numbers. To ignore alpha / beta / whatever versions you have to write a regex expression in a separate file and import that to the pom.xml. Of course you write the import statement in the configuration section of the plugin. Writing the rules in there? Naaaaah, too easy my friend. Extra file please.
    • Commands. There’s no help. There’s only “plugins”. No starting point, no idea what’s “default” and what the 1, 2, 3 steps are to get your app built. Just a bunch of horseshit you try out until your app is built. Or you think it is. (Or back to Stackoverflow, take the first example that works and go on to the next blackbox …)
    • Reposities. It seems I built my app several times correctly, then I configured some repositories, and suddenly it was built … more correctly. Don’t ask me why.

    Let’s continue with Spring. They once were (and probably still are) on the forefront of “modern”, modularized, reusable, enterprise-pattern-enabled, lightweight Java. Problem is, that’s still way too complex for any sane human being.

    Examples? Sure.

    • Configuration. I try to use Apache Camel. Seems to be a great product, all is configurable using either Spring DSL or code. Now if you want to do it with Spring, until today I invested days to find out how to read the fucking configuration from that god-damned configuration file. It’s just. Not. Possible. Sure, countless examples on the net, but that leads me to …
    • the documentation. Just some random notes:
      • The examples there are just don’t cut it and come basically without any documentation.
      • All links which sound like they could help lead to a 404 page.
      • “if you want to do X you can ‘easily’ do this using Spring XML” is a sentence I want to burn now, because … well, there’s no example, usually. You’re just being redirected to some Spring page which explains the Spring concepts in epic detail, with no connection whatsoever to the actual problem with the actual fucking library.
      • If they include a code snippet it’s always without any context. So you have the critical 3 lines, but if you want to include them into an actual application, you wonder about imports, namespaces, setup boilerplates, and countless other things every. Single. Time.
      • In summary it seems the docs are there for the pros who want to just quickly look something up they inconveniently forgot cause they already know a shit ton of shit. If you’re new to this? Yeah, go ahead, it’s easy, here’s a link to the Spring concepts.
      • Oh yeah, and it might be it’s just plainly, utterly, horribly wrong. But Java being nothing but enterprise, it’s at lease consistently wrong all the time.

    I think I managed to solve the problem just now. Unfortunately this cost me days, and only because of the most brainwashed, brainfucked eco-system I can imagine.

    (Disclaimer: I think Java has some really cool parts as well, just right now – I honestly don’t care. At all.)

     
    • oscerd 06:21 on 2020-04-14 Permalink | Reply

      All your findings about the first experience with Apache Camel may be really useful if reported to the users mailing list or dev mailing list. It would be valuable improve this experience from a documentation perspective. So, if you have time, it would be nice if you report back to the community mailing list.

  • penguin 19:27 on 2020-03-29 Permalink | Reply
    Tags: data sources, sars-cov-2   

    SARS-CoV-2 (“Corona”) Data Sources and APIs 

    Last update: 2020-04-01

    You just have to write about something, and it changes. In this case, that’s pretty good 🙂 . So here’s a list of APIs and data sources for the CoV-2 pandemia.

    APIs:

    Data sources:

    Dashboards and visualizations:

     

     
  • admin 17:35 on 2020-03-29 Permalink | Reply
    Tags: , pandas, , twitter   

    SARS-CoV-2, a.k.a “Corona Virus”, pseudo “data science” and Twitter 

    Some terminology (taken from – who would have thought it – t-online.de) and background information:

    • SARS-CoV-2 is the correct name of the virus. Everybody calls him “the Corona Virus” though, which is technically incorrect, cause there’s only the family of corona viruses – it’s a group, not a single one.
    • COVID-19 is the name of the disease. It’s an acronym made of “COrona VIrus Disease 19“. (It’s basically the same as HIV and AIDS – one is the infection, the other the actual illness).
    • It’s not the flu, but I think everybody has that down by now
    • It’s R0-value (how many people are infected by one person on average) is about 3 (source: RKI Germany)

    All of us are following the development of this very closely, cause we have no choice since we’re all basically locked at home. And even if we would go out everything’s closed. That gives us a ton of time to play DOOM Eternal (me), or write twitter bots who regularly publish the numbers of COVID-19 infections in Germany (me as well). As macabre as it is, this is very good oppotunity to start fooling around with data science. But to do this you need datasets, which are surprisingly hard to get. For the current case numbers  I found those:

    There’s probably more, but like I said – it’s surprisingly hard to find regularly updated, publicly available data sets in machine-readable form. (Also, this is the first time I go looking for this stuff, so maybe I just have no clue).

    Now back to the twitter bot. What does it do exactly? Go look for yourself, but if you are too lazy:

    • It prints the current infection rate at 8h, 12h, 16h, 20h. At the last time it will include a graph, a 1-week-forecast and a small evaluation how are stand today in comparison to the forecast which would have been made a week ago.
    • A friend helped me greatly by providing first a Holt-Winters prediction (which is live now), and then upgraded this to a more intelligent ARIMA prediction, which is still buried in a jupyter notebook and waiting for daylight. (Maybe she will write a guest post here to explain what it does and how it works – but I haven’t asked yet).
    • As for the bot’s code – it’s not (yet) public. Which is unusual for me, really. I should remedy that.

    What I learned so far:

    • It’s surprsingly easy and hard at the same time to get a Twitter developer account
    • Twitter does not permit publishing the exact same tweet multiple times in a short period of time
    • Heroku is really really nice for this, as long as you don’t need to pay for it. I would be interested in alternatives.
    • matplotlib is a lot more complicated than I expected
      • but strangely neither bokeh, nor plotly can actually export png graphics without either a separate electron app (WTF?) or a headless browser and selenium (W-T-F?!?)
    • pandas rocks, or more precisely: pandas data frames rock.
    • there does not seem to be a single properly maintained Twitter library for JavaScript, but there is at least tweepy for Python. (I mean I want to try something in JS, but honestly, if everything I find is outdated, I just stay with good old Python …)
    • The infection growth is intensely exponetial – almost a straight line on a log scale plot.

    Let’s see where this goes, and I hope you all stay healthy.

     
  • admin 08:32 on 2020-03-22 Permalink | Reply
    Tags: ,   

    Configure Python on Windows 

    All right, I have a Windows machine. It’s a PITA, but it’s here. And for some reason I started doing some Python testing on it. So this is how I managed to do it:

    Preparation:

    • Install python with choco (choco install -y python)
    • Run PowerShell as Administrator
      • Execute Set-ExecutionPolicy -ExecutionPolicy Unrestricted (we’ll see why in a very short time)

    Now to code it’s pretty similar to *NIX:

    • Create your code folder
    • Set up a python venv (python -m venv .env)
    • In VS Code, choose this interpreter

    So why the PowerShell stuff? Cause to activate the environment VS Code needs to execute a .ps1 script. Which it can’t, cause “executing scripts is disabled on this machine”, which seems to be the default setting.

    All in all, surprisingly straightforward. And I just noticed even the *NIX keyboard shortcuts (CTRL-A, CTRL-K, for example) work in the terminal window now. Crazy.

     
c
compose new post
j
next post/next comment
k
previous post/previous comment
r
reply
e
edit
o
show/hide comments
t
go to top
l
go to login
h
show/hide help
shift + esc
cancel