Stargate Logo  Secured by phpPhobos

Smurphtools   
  Login  - No Account?  Create One   
Search 
Thursday, 9 July 2020  -
News FeedRSS Feed
rss rdf  
  Home 
  Authentication 
 Documentation 
    Site Documentation 
    About me 
  Legal Notice 
 Applications 
    Web Links 
 Hobbies 
    RC Models 
    RC Batteries 
    Build blogs 
 Modules 
    Downloads 
    Weblinks 
 Blacklists 
    Blacklist 
    Blockout 
    DNS Blacklist 
 Registered Bloggers 
  Blog de Maiwenn 
  Blog de Morgane 
  Joerg's Blog 
 Gallery [Listing]
  > Diving 
  > Martinique 
  > Steampunk 
  > RC Planes 
  > FPV Drones 
  > Indy travels 
  > Budapest 
 FAQ  [ Topics  ]
 Common Linux problem... 
 Routerboard RBxxxAH 
 Apple Mac-mini 
 PHP Phobos 
 Stargate's Backup sc... 
 eBook Reader / PRS-5... 
 RC Models 
 Server in SolLan 
 System Stats [ Index ]
 Router
  > Gateway 
 Server
  > Stargate 
  > Halley 
 Client
  > RemPi 
 Weather Station
  > WeatherPI 
Build blog - export: raw
 
Documentation TitleTitle of this documentation entry
:  
  Cleanup the Raspberry PI OS: Raspbian  
 
Document AuthorPerson who wrote this entry
:  
  Jörg Mertin  
 
Contact E-MailContact E-Mail for this document
:  
  pizerow_cleanup(-AT-)solsys(-DOT-)org  
 
StatusStatus for this entry
:  
  In Progress  
 
CategoryCategory/Type of this entry
:  
  Electronics  
 
Document languageLanguage this document has been written
:  
  English  
 
Entered dateEntered date
:  
  2020-01-08  
  Parent:   1 - Weather Station
Main document

Introduction

I decided to go with a raspberry Pi Zero W for various of the projects I have (Media player, Weather station etc.). However, RaspBian comes with too many programs loaded, and very often, the SD Card dies because of too many writes all the time. Reason I started to document what I do to remove all the unneeded running crap for my projects.

The below information has been gathered from various places on the internet, and from own experience with the Raspberry PI models I use since release B+.
As I use snippet of information from here and there, I will not always write down all sources. But some will be there.

Performing an minimal installation

I used etcher to install the Raspbian Buster Lite. Follow the guide on the previously provided Etcher link to put it on the SD-Card.
Note: The reason I use the lite release is that most of the software that will be installed by a regular release is Desktop based. On a media-center or a weather station, that is only dead weight, and whatever is not used is a potential security flaw.

As I tend to do all remote, I have to configure 2 things on first boot

  1. WiFi for remote access
    Create a file called wpa_supplicant.conf and place it into the /boot partition on the SD-Card. Add the following content:

    network={
     ssid="WiFi SSID"
     psk="WiFi Password"
    }
  2. SSH Damon to be activated
    That one is easy, just place an empty file called ssh into the /boot partition on the SD-Card. Do it with:

    ~# touch ssh

    The ssh file will be removed after the first boot. So if you want to keep ssh running after a reboot, you will have to enable it on first boot in systemd with:

    ~# sudo systemctl enable ssh

Using raspi-config for various tasks

the raspbian setup by default configures a graphical UI and uses 128MBytes of Ram from the system for the GUI. This is lost memory, as using the Raspberry PI zero W, you usually want to have a little application running it for media-center or Weather station.
Start raspi-config with sudo raspi-config and:

  • go to "Boot Options" and select "1 Desktop / CLI" => "B1 Console"
  • go to "Advanced Options" and select "A3 Memory Split", and set the split to 32Mb. Note that 16Mb seems to not work. So 32 is the smallest we can set.

WARNING: If you use the SkyCam in the weather station, set it to 128 or the Camera will not work!

This should provide your software some 112MBytes more Ram. Which on such tiny device, is always welcome.
Check it out with:

pi@raspberrypi:~ $ free
              total        used        free      shared  buff/cache   available
Mem:         475304       73700      296768        3416      104836      347464
Swap:             0           0           0

If you don't have at leasat 475Mbytes on the Total column, try the next highest value. NOTE: In case you run a Camera top take pictures, leave it to 128M or else the camera won't work.

Removing all non needed drivers and services

Removing bluetooth drivers

This actually disables the entire BlueTooth functionality on the PI.

Create a file /etc/modprobe.d/raspi-blacklist.conf with the following content:

# Bluetooth
blacklist btbcm
blacklist hci_uart

Then make sure you disable that service in systemd with

~# sudo systemctl disable bluetooth
~# sudo systemctl stop bluetooth
~# sudo systemctl disable hciuart.service

Next time you reboot, the fact that the drivers and services won't be loaded will save memory.

Removing alsa drivers

Create a file (if not already created) /etc/modprobe.d/raspi-blacklist.conf and add the following content.

# Alsa & Audio
blacklist snd_bcm2835

Alsa drivers are used for audio control, which is not required on a non-media device. So if you want to remove these (Weather station for example), follow this guide:

~# sudo systemctl disable alsa-utils

Removing alsa-utils will not really work, as it will also remove raspi-config due to dependency constraints. We can leave the package there.

Disable non needed services

  • Disable avahi
    The Pi is going to use simple uni-cast DNS - multi-cast DNS support is not required.

    ~# sudo systemctl disable avahi-daemon
    ~# sudo systemctl stop avahi-daemon
  • Disable TriggerHappy
    The Pi isn't being used with button. Disable the TriggerHappy daemon (not that it is a init.d service)

    ~# sudo systemctl disable triggerhappy
    ~# sudo systemctl stop triggerhappy 

Reducing the writes onto the SD-Card

Suppress chatiness of some services

Suppress GnuPG entries from systemd and gpg-agent

Create the file /etc/rsyslog.d/ignore-systemd-gpg.conf and add the following content:

if $programname == "systemd" and ($msg contains "Listening on GnuPG" or $msg contains "Closed GnuPG") then stop

After a restart or issuing the below command, these message floodings will be gone. Note - if you don't log in often on the machine, leave it as is. It will show you when people tried to actually log in.

~# sudo /etc/init.d/rsyslog restart

Suppress ntpd sollicication entries from systemd

Create the file /etc/rsyslog.d/ignore-systemd-ntpd.conf and add the following content:

if $programname == "ntpd" and ($msg contains "Soliciting pool server") then stop

After a restart or issuing the below command, these message floodings will be gone.

~# sudo /etc/init.d/rsyslog restart

Remove SWAP

Swap is a virtual memory extension written to disk. Actually very bad for the SD-Card on our Raspberry PI. The most common cause or SD_Card failure is too many writes. And if the PI is to SWAP, would kill the SD-Card pretty fast. So let's remove it.

As not every method always worked (from experience), I managed to use the below steps to disable it.

  1. Set CONF_SWAPSIZE=0 in /etc/dphys-swapfile file.

  2. Disable and remove the dphys-swapfile subsystem

    ~# sudo dphys-swapfile swapoff
    ~# sudo dphys-swapfile uninstall
    ~# sudo update-rc.d dphys-swapfile remove
    ~# sudo apt purge dphys-swapfile

Optimize log-file writes to the SD-Card by only writing data over once every 3 hours.

source: My solution for Raspbian 8.0 (Jessie) based on logging to RAM

We will export the current/regular log-file writes to be done on a Ram-filesystem.
For this, add the following line to /etc/fstab:

tmpfs           /var/tmp        tmpfs   size=10M,nodev,nosuid     0       0

We limite this temporary filesystem to 10MBytes.

The log2disk script

We need to save the below script in /usr/local/bin/log2disk which will append and delete the contents from all log files in /var/tmp/log/ to the files in /var/log/.

#!/bin/sh

# Author: Frank Breitling 
DESC="Moving contents from /var/tmp/log/ to /var/log/"

if [ $(id -u) -ne 0 ]
then echo "Please run as root"
     exit
fi

echo $DESC

exec >>/var/log/log2disk.log 2>&1

date
cd /var/tmp/

for i in log/*; do
    basename $i
    cat $i >>/var/$i
    >$i
done

and make it executable with:

sudo chmod +x /usr/local/bin/log2disk

Adding log2disk to crontab

We want to run this script every 3 hours and add this line to the system's /etc/crontab. Run crontab -e as root user, and you're editing it for root straight.

10 */3  * * *   /usr/local/bin/log2disk

(Don't forget a final newline which is needed by crontab.)

Installing the log2disk.service

We need to create a systemd service in /lib/systemd/system/log2disk.service that executes this script before shutdown and reboot, so that the log file contents gets preserved:

[Unit]
Description=Write log files to disk
RequiresMountsFor=/
Before=rsyslog.service

[Service]
Type=oneshot
RemainAfterExit=true
ExecStart=/bin/true
ExecStop=/usr/local/bin/log2disk

[Install]
WantedBy=multi-user.target

and install it with

sudo systemctl enable log2disk

Selecting the log files for RAM

Now we can tell /etc/rsyslog.conf which logfiles to keep in RAM. These files are auth.log, syslog, daemon.log, user.log and messages and we replace for each of their entries the log/ path by tmp/log/ for example like this:

auth,authpriv.*                 /var/tmp/log/auth.log

Add the NTP Daemon

The NTP daemon will make sure the time on your PI is always up to date. Even if you have a local clock/addon board, the internet time will be there to have the correct time configured.
Add the ntp-daemon with:

~# sudo apt install ntp

Edit the file /etc/ntp.conf add your own ntp server if you have one. I use my home-server and the default debian pool servers.
Enable and start the ntpd with:

~# sudo systemctl start ntp
~# sudo systemctl enable ntp

Disable ipv6 (if not needed)

As my raspberry PI's are always running inside my LAN, I tend to disable whatever is not required. And IPv6 is one of these protocols that is disabled by default in my LAN, and will get dropped by my firewall anyway.
So - for IPv6 to be disabled permanently on my LAN, I disable it with adding the following to the /etc/sysctl.d/disableipv6.conf:

# Disable IPv6 on all interfaces
net.ipv6.conf.all.disable_ipv6=1
# Disable IPv6 autoconfiguration on all interfaces
net.ipv6.conf.all.autoconf=0
# Disable IPv6 on newly added interfaces
net.ipv6.conf.default.disable_ipv6=1
# Disable IPv6 autoconfiguration on newly added interfaces
net.ipv6.conf.default.autoconf=0

And - to save memory, we can also prevent the IPv6 kernel module to load with adding the below content to the file /etc/modprobe.d/ipv6.conf:

# Don't load ipv6 by default
alias net-pf-10 off
#alias ipv6 off
# added to disable ipv6
options ipv6 disable_ipv6=1
# added to prevent ipv6 driver from loading
blacklist ipv6

Conclusion

Finally reboot the PI and let it run for a while. If all runs fine, you'll see the log-files being dumped onto the disk once every 3 hours.
Also, the system should be loaded, using very few memory and ready to receive the software it is supposed to receive.

The following apps will run:

systemd-+-agetty
        |-cron
        |-dbus-daemon
        |-dhcpcd
        |-ntpd---{ntpd}
        |-polkitd---2*[{polkitd}]
        |-rngd---3*[{rngd}]
        |-rsyslogd---3*[{rsyslogd}]
        |-sshd---sshd---sshd---bash---pstree
        |-systemd---(sd-pam)
        |-systemd-journal
        |-systemd-logind
        |-systemd-udevd
        `-2*[wpa_supplicant]

This is what the system resources will look like:

              total        used        free      shared  buff/cache   available
Mem:         491796       30304      395612        3488       65880      408912
Swap:             0           0           0
History
  • Thu Jan 9 2020, first version of document done.
  Last change:     2020-04-17 22:17:40  
    Back Back    

 
 
Problems to  root(-AT-)solsys(-DOT-)org  - best viewed @ 1920bpp
This site is powered by phpPhobos v2.0b446
© J. Mertin smurphy(-AT-)solsys(-DOT-)org 
Icons - Copyright Breeze artists GPL 2+