Updates from May, 2020 Toggle Comment Threads | Keyboard Shortcuts

  • penguin 10:23 on 2020-05-06 Permalink | Reply
    Tags: ,   

    Homebrew, OpenSSL and PowerShell 

    On Mac, PowerShell really, really wants to specifically use OpenSSL version 1.0.

    Unfortunately, homebrew switched to OpenSSL version 1.1 in v2.2, because OpenSSL 1.0 is end-of-life.

    This fixes it (for now at least):

    brew uninstall openssl --ignore-dependencies
    brew uninstall openssl --ignore-dependencies
    brew install https://github.com/tebelorg/Tump/releases/download/v1.0.0/openssl.rb

    References:

     
  • penguin 12:24 on 2020-04-26 Permalink | Reply
    Tags: , ubuntu, updates   

    Ubuntu update procedure 

    Brain dump.

    # from here: https://askubuntu.com/a/862799/870970
    apt-get update && apt-get --with-new-pkgs upgrade 
    
    # restart
    reboot
    
    # do it again
    apt-get update && apt-get --with-new-pkgs upgrade 
    
    # remove shit
    apt autoremove
    
    
    #
    # UPGRADE
    # 
    
    do-release-upgrade 
    
    
    # done.

     

     
  • penguin 23:18 on 2020-04-25 Permalink | Reply
    Tags: , , , nextcloud   

    nextcloud and Docker and reverse proxies 

    I have a nextcloud setup like described here (docker-compose, let’s encrypt proxy companion, postgres and nextcloud). And for a while I couldn’t connect any new nextcloud clients to the installation.

    This fixed it:

    <?php
    $CONFIG = array (
      # manually added because it's not picked up from
      # the env vars once set ... it seems ...
    
      # the docker IP range
      'trusted_proxies' => ["172.16.0.0/12"],
    
      # the hostname of the server
      'overwritehost'   => "my.super.secret.server",
    
      # the ENDUSER->PROXY protocol, NOT the proxy-> nextcloud protocol!
      'overwriteprotocol' => "https",
    
      #
      # AAAND NOW back to the original config file ...
      # ...
    )

    Some notes:

     
  • admin 08:32 on 2020-03-22 Permalink | Reply
    Tags: ,   

    Configure Python on Windows 

    All right, I have a Windows machine. It’s a PITA, but it’s here. And for some reason I started doing some Python testing on it. So this is how I managed to do it:

    Preparation:

    • Install python with choco (choco install -y python)
    • Run PowerShell as Administrator
      • Execute Set-ExecutionPolicy -ExecutionPolicy Unrestricted (we’ll see why in a very short time)

    Now to code it’s pretty similar to *NIX:

    • Create your code folder
    • Set up a python venv (python -m venv .env)
    • In VS Code, choose this interpreter

    So why the PowerShell stuff? Cause to activate the environment VS Code needs to execute a .ps1 script. Which it can’t, cause “executing scripts is disabled on this machine”, which seems to be the default setting.

    All in all, surprisingly straightforward. And I just noticed even the *NIX keyboard shortcuts (CTRL-A, CTRL-K, for example) work in the terminal window now. Crazy.

     
  • penguin 12:00 on 2020-03-13 Permalink | Reply
    Tags: autohotkey,   

    autohotkey 

    Under macOS I use TextExpander, under Windows there’s the fantastic AutoHotkey. One of the few softwares I can’t live without.

    This is my default configuration:

    ; ---------- "auto reload" ----------
    
    FileGetTime ScriptStartModTime, %A_ScriptFullPath%
    SetTimer CheckReload, 1000, 0x7FFFFFFF ; ms & priority
    
    ; from here: https://stackoverflow.com/a/45488494
    CheckReload() {
        global ScriptStartModTime
        FileGetTime curModTime, %A_ScriptFullPath%
        If (curModTime <> ScriptStartModTime) {
            Loop
            {
                reload
                Sleep 300 ; ms
                MsgBox 0x2, %A_ScriptName%, Reload failed. ; 0x2 = Abort/Retry/Ignore
                IfMsgBox Abort
                    ExitApp
                IfMsgBox Ignore
                    break
            } ; loops reload on "Retry"
        }
    }
    
    
    ; ---------- actual content here ----------
    
    ; removed all my email address shortcuts ...
    
    :*:-dt::
      ; from here: https://is.gd/3u6MKQ
      Send, %A_YYYY%-%A_MM%-%A_DD%
      Return
    
    :*://ts::
      Send, %A_YYYY%%A_MM%%A_DD%_%A_Hour%%A_Min%%A_Sec%
      Return

     

     
  • penguin 21:28 on 2020-01-25 Permalink | Reply
    Tags: check_mk, , ,   

    Check MK container/k8s deployment 

    In the company everybody seems to love Check MK. Me? Not so much, but a better alternative costs time and effort, both resources we don’t really have right now. Yet there’s a positive thing about it – because there’s an official docker container. Since I already coded a helm chart for stateful single container softwares (which I personally find super useful), I just wrote a Check MK YAML and installed it on my K8S cluster.

    And then nothing worked. Turns out, Apache – which is used in that very strange “Open Monitoring Distribution” which Check MK seems to have been at one point – has a slightly sub-optimal configuration for running in a container behind a load balancer using cert-manager.

    In short, you connect to the load balancer using “cmk.my.domain”, and it redirects you to the container port, which to itself is “https://cmk.my.domain:5000/&#8221; and just wrong. Which brings me to the question if anybody has ever tried to run the Check MK container in a k8s cluster or behind a load balancer, which brings me to the question that I’d rather use software which actively embraces that, which brings me to the question WHICH ONE?!? which brings us back to “no resources, no time”.

    So, bad luck, Check MK it is. But what about the bug? Reporting it you get an email “DONT CALL US – WE CALL YOU (and we probably won’t)“, with a ticket ID but no link. So probably no help here. So I “forked” the container, fooled around with it, and found a solution. The “fixed” container is now available on docker hub (sources on GitHub) and running nicely in our internal cluster. Let’s see which hidden bugs I have introduced 😉 . The stasico-Helm-YAML file I used to deploy Check MK in K8S is also available.

    TL;DR
     
  • penguin 09:50 on 2018-05-25 Permalink | Reply
    Tags: , , rbac   

    Helm in a kops cluster with RBAC 

    I created a K8S cluster on AWS with kops.

    I ran helm init to install tiller in the cluster.

    I ran helm list  to see if it worked.

    I got this:

    Error: configmaps is forbidden: User "system:serviceaccount:kube-system:default" \ 
        cannot list configmaps in the namespace "kube-system"

    That sucked. And google proved … reluctant. What I could figure out is:

    Causes

    • kops sets up the cluster with RBAC enabled (which is good)
    • helm (well, tiller) uses a standard role for doing things (which might be ok, at least it was with my stackpoint cluster), but in that case (for whatever reason) it did not have sufficient privileges
    • so we need to prepare some cluster admin roles for helm to use

    Fixes

    Just do exactly as it says in the helm docs 🙂 :

    • apply the RBAC yaml file which creates the kube-system/tiller service account, and binds this to the cluster-admin  role.
    • install helm with: helm init –service-account tiller

    Is that secure? Not so much. With helm you can still do anything to the cluster at all. I might get to this in a later post.

     
  • penguin 18:10 on 2018-04-24 Permalink | Reply
    Tags: , gitlab   

    GitLab spot runners & Puppet 

    We are on AWS with GitLab. For ease of use, and because our build hosts degenerate for some reason (network issues), we decided to use spot instances with GitLab.

    The journey was all but easy. Here’s why.

    GitLab Runner configuration complaints

    First: The process

    To configure GitLab runner, you have to …

    • install GitLab,
    • write down the runner registration token,
    • start a runner,
    • manually a registration command using above token.

    That registration command will then modify the config file of the runner. That is important because you can’t just write a static, read-only config file and start the runner. This is not possible for two reasons:

    • when you execute the registration command, the runner wants to modify the config file to add yet another token (its “personal” token, not the general registration secret), so it must not be read-only
    • the runner has to be registered, so just starting it will do … nothing.

    That is in my eyes a huge design flaw, which undoubtedly has its reasons, but it still – sorry – sucks IMHO.

    Second: The configuration

    You can configure pretty much everything in the config file. But once the runner registers, the registration process for some reason appends a completely new config to any existing config file, so that … the state is weird. It works, but it looks fucked, and feels fucked.

    You can also set all configuration file entries using the gitlab-runner register  command. Well, not all: The global parameters (like, for example, log_level  or concurrent ) cannot be set. Those have to be in a pre-existing config file, so you need both – the file and the registration command, which will look super ugly in a very short time.

    Especially if you still use Puppet to manage the runners, cause then you just can’t just restart the runner once the config file changes. Because it will always change, because of above reasons.

    Third: The AWS permission documentation

    Another thing is that the list of AWS permissions the runner needs in order to create spot instances is nowhere to be found. Hint: EC2FullAccess  and S3FullAccess is not enough. We are using admin permissions right now, until we figured it out. Not nice.

    Our solution

    For this we’re still using Puppet (our K8S migration is still ongoing), and our solution so far looks like this:

    • Create a config file with puppet next to the designated config file location,
      • containing only global parameters.
      • The file has a puppet hook which triggers an exec that deletes the “final” config file if the puppet-created one has changed.
    • Start the GitLab runner.
    • Perform a “docker exec” which registers the runner in GitLab.
      • The “unless” contains a check that skips execution if the final config file is present.
      • The register  command sets all configuration values except the global ones. Like said above, the command appends all non-global config settings to any existing config file.

    Some code

    #
    # CONFIG VALUES
    #
    
    # the configuration for the build runner running on the same host, which
    # manages the autoscaling-based spot-instance-allocation
    global::concurrent:                       '5'
    
    registration_command::docker_image:       'ubuntu:artful'
    registration_command::token:              'our-token'
    registration_command::url:                'https://our.gitlab.server'
    
    runners::concurrency:                     '1'
    runners::limit:                           '10'
    runners::name:                            'spot-runner'
    
    cache::bucket_location:                   'eu-central-1'
    cache::bucket_name:                       'our-cache-bucket'
    cache::shared:                            'true'
    cache::type:                              's3'
    
    machine::idle_nodes:                      '0'
    machine::idle_time:                       '1800'
    machine::max_builds:                      '10'
    machine::machine_name:                    'standard-%s'
    
    machine::option::access_key:              "%{hiera('ops::gitlab::spotrunner::id')}"
    machine::option::ami:                     'ami-44d48eaf'
    machine::option::block_minutes:           '60'
    machine::option::iam_profile:             'gitlab-runner'
    machine::option::instance_type:           'm4.xlarge'
    machine::option::private_address_only:    'true'
    machine::option::private_address:         'true'
    machine::option::region:                  'eu-central-1'
    machine::option::secret_key:              "%{hiera('ops::gitlab::spotrunner::secret')}"
    machine::option::spot_instances:          'true'
    machine::option::spot_price:              '0.2'
    machine::option::subnet_id:               'subnet-subbb'
    machine::option::tags:                    'stage,prod'
    machine::option::vpc_id:                  'vpc-veepeecee'
    
    machine::option::other:                   >
      --machine-machine-options amazonec2-security-group=cognodev3_sg_docker_machine
      --machine-machine-options amazonec2-security-group=cognodev3-SG0-shubdu
      --machine-machine-options amazonec2-security-group=cognodev3-SG1-shalala
      --machine-machine-options amazonec2-security-group=cognodev3-SG2-shubndibndu
      --machine-machine-options engine-insecure-registry=some.internal.repo
      --machine-machine-options engine-insecure-registry=other.internal.repo:5000
    
    
    
    #
    # START
    #
    
    ## GITLAB SPOT INSTANCE RUNNER
    
    create::docker_containers:
      # gitlab spot runner
      gitlab-spot-runner:
        image:              gitlab/gitlab-runner:latest
        pull_on_start:      true
        volumes:
          - /var/docker-apps/gitlab-spot-runner:/etc/gitlab-runner
          - /var/run/docker.sock:/var/run/docker.sock
          - /var/run/.docker:/root/.docker
    
    create::files:
      '/var/docker-apps/gitlab-spot-runner/config.toml.puppet':
        ensure: present
        notify: 'Service[docker-gitlab-spot-runner]'
        content: |
          concurrent = %{hiera('global::concurrent')}
          check_interval = 0
    
    
    create::execs:
      'delete config file':
        command:     rm /etc/gitlab-runner/config.toml
        path:        /bin:/usr/bin:/sbin:/usr/sbin
        refreshonly: true
        subscribe:   File[/etc/gitlab-runner/config.toml.puppet]
        before:      Docker::Exec[register-spot-runner]
    
    # why are we doing this?
    # because gitlab WANTS to re-write the config file. and we CANNOT set "global"
    # parameters (e.g. "log_level", "concurrent") in using the registration
    # command line.
    # so, we ...
    #    * create a config "config.toml.puppet" file with ONLY global params
    #    * rm the config.toml file in case of changes to the .puppet file
    #    * re-execute registration using ALL other config settings when the
    #      config.toml file is no longer present
    # we do this because:
    #    * we can't watch the toml file for changes, cause the runner WILL change
    #      it
    #    * but we WANT to set global parameters, and luckily the runner seems
    #      to just append all non-global settings to an existing config file
    #      on registration.
    # THIS FUCKING SUCKS. I know. but I really have not found another way (abk)
    
    create::docker_execs:
      'register-spot-runner':
        detach:     false
        container:  gitlab-spot-runner
        tty:        true
        command: >-
          /bin/bash -c 'cp /etc/gitlab-runner/config.toml.puppet /etc/gitlab-runner/config.toml &&
          gitlab-runner register --non-interactive
          --registration-token %{hiera('registration_command::token')}
          --url %{hiera('registration_command::url')}
          --executor "docker+machine"
          --docker-image %{hiera('registration_command::docker_image')}
          --locked=false
          --run-untagged
          --docker-volumes /var/run/docker.sock:/var/run/docker.sock
          --name %{hiera('runners::name')}
          --limit %{hiera('runners::limit')}
          --request-concurrency %{hiera('runners::concurrency')}
          --cache-cache-shared=%{hiera('cache::shared')}
          --cache-s3-bucket-location %{hiera('cache::bucket_location')}
          --cache-s3-bucket-name %{hiera('cache::bucket_name')}
          --cache-type %{hiera('cache::type')}
          --machine-machine-driver amazonec2
          --machine-idle-nodes %{hiera('machine::idle_nodes')}
          --machine-idle-time  %{hiera('machine::idle_time')}
          --machine-machine-name %{hiera('machine::machine_name')}
          --machine-max-builds %{hiera('machine::max_builds')}
          --machine-machine-options amazonec2-private-address-only=%{hiera('machine::option::private_address_only')}
          --machine-machine-options amazonec2-use-private-address=%{hiera('machine::option::private_address')}
          --machine-machine-options amazonec2-region=%{hiera('machine::option::region')}
          --machine-machine-options amazonec2-access-key=%{hiera('machine::option::access_key')}
          --machine-machine-options amazonec2-ami=%{hiera('machine::option::ami')}
          --machine-machine-options amazonec2-block-duration-minutes=%{hiera('machine::option::block_minutes')}
          --machine-machine-options amazonec2-iam-instance-profile=%{hiera('machine::option::iam_profile')}
          --machine-machine-options amazonec2-instance-type=%{hiera('machine::option::instance_type')}
          --machine-machine-options amazonec2-request-spot-instance=%{hiera('machine::option::spot_instances')}
          --machine-machine-options amazonec2-secret-key=%{hiera('machine::option::secret_key')}
          --machine-machine-options amazonec2-spot-price=%{hiera('machine::option::spot_price')}
          --machine-machine-options amazonec2-subnet-id=%{hiera('machine::option::subnet_id')}
          --machine-machine-options amazonec2-tags=%{hiera('machine::option::tags')}
          --machine-machine-options amazonec2-vpc-id=%{hiera('machine::option::vpc_id')}
          %{hiera('machine::option::other')}'
        unless:     test -f /etc/gitlab-runner/config.toml
        require:    Service[docker-gitlab-spot-runner]
    

    Does this look ugly? You bet.

    Should this be a puppet module? Most probably.

    Did I foresee this? Nope.

    Am I completely fed up? Yes.

    Is this stuff I want to do? No.

    Does it work?

    Yes (at least … 🙂 )

    Remarks

    If you wander what all those create::THING  entries are – it’s this:

    $files = hiera_hash('create::files', {})
    if ! empty($files) {
      create_resources('file', $files)
    }
    

    We have an awful lot of those, cause then we can do a lot of stuff in the config YAMLs and don’t need to go in puppet DSL code.

     
  • penguin 08:07 on 2018-02-23 Permalink | Reply
    Tags: font,   

    Ugly ligatures in Linux 

    Unfortunately boohomil went off grid. I still haven’t replicated his config fully. And it still sucks.

    One more step was fixing those super-ugly ligatures in Linux. Works at least in LibreOffice (just restart the app to see changes).

     
  • penguin 08:55 on 2018-02-01 Permalink | Reply
    Tags: commandline,   

    crontab and nano 

    Ever used update-alternatives to switch everything to vim and … crontab -e still used nano?

    Well, I had this. I found the answer:

    select-editor

     

     
c
compose new post
j
next post/next comment
k
previous post/previous comment
r
reply
e
edit
o
show/hide comments
t
go to top
l
go to login
h
show/hide help
shift + esc
cancel