Category: Software Development

Well, since you are here, you are probably planning to spy on traffic generated by the apps on your phone, probably looking to find out what apps are leaking your private information !

How this all works: When an application on your phone contacts a website/web-server over a secure connection, your phone checks the certificate, and checks whether or not that certificate was issued by an authority the phone trusts, what we are doing is telling the phone to add YOU to the list of trusted authorities, then, we create a proxy that the phone connects through (mitmproxy). Now, the proxy contacts the website on behalf of the application that sent the request, gets the response / contents of that page, saves a copy of that response, and repackages it using our own security certificate, then sends it to the phone, at this stage, the phone thinks the certificate belongs to the website it was contacting (Because now it trusts you), and everything works as it normally would. This does not work if the app has a pinned certificate, a pinned certificate is when an app does not trust the list of authorities the system maintains and requires a certain certificate, but this too has a workaround !

here is what you need to do.

I am assuming you already have a rooted virtual android device (AVD), if not, here are the instructions, in my case, even though it is mid 2026, i decided to go with a Google Pixel 6 with android 12 (The pixel six has updates up to the most current Android 17, but i decided to go with Android 12)

You need to create a root certificate, and install it in both mitmproxy and android, the mitmproxy gets the pem file with the certificate and private key (So that it can create certificates for websites on the fly), while android gets the one with the certificate (Obviously)

1- Gnerate the private key

openssl genrsa -out android-mitm-ca.key 4096

2 – Create the CA certificate

openssl req -x509 -new -sha256 -key android-mitm-ca.key -out android-mitm-ca.crt -days 3650 -subj "/C=JO/O=Android Traffic Analysis/CN=Android MITM Analysis CA" -addext "basicConstraints=critical,CA:TRUE" -addext "keyUsage=critical,keyCertSign,cRLSign"

3- Installing mitmproxy (On your debian machine)

install Prerequisites:

sudo apt install python3-dev libffi-dev libssl-dev libxml2-dev libxslt1-dev libjpeg-dev zlib1g-dev

Debian 13 does not have it in repositories (12, and 14 do !), so we need to install from binaries downloaded from the website ! so i put them in /opt/mitmproxy and symlinked the files in /usr/local/bin

4- Concatinate/combine key and cert into 1 file for the mitmproxy

cat android-mitm-ca.key android-mitm-ca.crt > ~/.mitmproxy/mitmproxy-ca-my.pem
chmod 600 ~/.mitmproxy/mitmproxy-ca-my.pem

5- running and check that mitmproxy is using our cert

First, run mitmproxy.

NO-GO: mitmproxy –listen-host 192.168.7.119 –listen-port 8080 –certs “*=$HOME/.mitmproxy/mitmproxy-ca-my.pem” (No go because certs is for individual domains)

NO-GO:

mitmproxy --certs "*=$HOME/.mitmproxy/mitmproxy-ca-my.pem"

Or if needed

mitmproxy --listen-host 192.168.7.119 --listen-port 8080 --set confdir="$HOME/.mitmproxy-my"

Or if you want access from outside your network

mitmproxy --listen-host 192.168.7.129 --listen-port 8080 --set block_global=false --set confdir="$HOME/.mitmproxy-my"

From another terminal, let us try to get the contents of example.com through the proxy:

openssl s_client -proxy 192.168.7.119:8080 -connect example.com:443 -servername example.com </dev/null 2>/dev/null | openssl x509 -noout -issuer -subject

Now, if the proxy works as intended, and is correctly signing our requests with our own certificate, we should see something like isuer: Android MITM Analysis CA (The name we chose)

So, now our proxy is ready and serving, all that remains is relevant to Android, we have already made the android certificate to add (OUR_HASH.0)

Installing the certificate on the android phone

The only thing that is confusing about this section is that not all android phones work in the same way ! I am deliberately using Android 12 (Will install most software while being somewhat easier to tinker with), you on the other hand may be using android 16 or 17 (The latest at the time of writing).

Also, I am using a virtual android phone (With Android Studio), you may be using an actual rooter physical phone !

So, I will outline when something is going to be different on other android versions, and you will have to figure it out (Use Google, or gemini or something)

6- Certificate File Name for android

figure out the hash for the filename of the Android file (The file name should be “hash.0”, replace hash with the actual hash resulting from the below) this is how android expects the file name to be

HASH=$(openssl x509 -inform PEM -subject_hash_old -in android-mitm-ca.crt | head -1)
echo "$HASH"

Now, copy the file android-mitm-ca.crt and name the copy the same as the resulting hash with an added .0 at the end (THEHASH.0)

7- If android 16+ (I am using 12, but if you have 16+)

If your rootAVD image is Android 16 or above, don’t blindly follow the old /system/etc/security/cacerts/ procedure. Android 16 changed the system certificate location, and current mitmproxy documentation has a separate procedure involving a Magisk certificate-moving module.

check your android version

adb shell getprop ro.build.version.release
adb shell getprop ro.build.version.sdk
adb shell getprop ro.product.cpu.abi
adb shell su -c 'id'
$ adb shell getprop ro.build.version.release
12
$ adb shell getprop ro.build.version.sdk
31
$ adb shell getprop ro.product.cpu.abi
x86_64
$ adb shell su -c ‘id’
uid=0(root) gid=0(root) groups=0(root) context=u:r:magisk:s0

So, in my case, I am running android 12 ! Software development kit 31 (API level), X86 cpu, and (context=u:r:magisk:s0)

8- Explore the filesystem

Let us check and explore the files and folders on filesystem

> adb shell su -c 'ls -ld /system/etc/security/cacerts'
drwxr-xr-x 2 root root 4096 2023-07-26 23:02 /system/etc/security/cacerts

> adb shell su -c 'mount | grep -E " /system | / "'
/dev/block/dm-4 on / type ext4 (ro,seclabel,relatime)

> adb shell su -c 'ls -l /system/etc/security/cacerts | head'
total 952
-rw-r--r-- 1 root root 2871 2023-07-26 23:02 01419da9.0
-rw-r--r-- 1 root root 2914 2023-07-26 23:02 04f60c28.0
-rw-r--r-- 1 root root 2345 2023-07-26 23:02 0d69c7e1.0
-rw-r--r-- 1 root root 4622 2023-07-26 23:02 10531352.0
-rw-r--r-- 1 root root 4724 2023-07-26 23:02 111e6273.0
-rw-r--r-- 1 root root 4308 2023-07-26 23:02 12d55845.0
-rw-r--r-- 1 root root 2599 2023-07-26 23:02 1ae85e5e.0
-rw-r--r-- 1 root root 4931 2023-07-26 23:02 1df5a75f.0
-rw-r--r-- 1 root root 4535 2023-07-26 23:02 1e1eab7c.0

So, we do have the path /system/etc/security/cacerts on the phone, system (See second command above) is mounted read only, there are 952 certificate files in the /system/etc/security/cacerts folder.

Reminder: On Android 12, the traditional system CA location we’re interested in is: /system/etc/security/cacerts/ , The certificates are stored there using the OpenSSL subject-hash filename convention, e.g.: xxxxxxxx.0

The plan at this stage is to use magisk to add our certificate as an overlay, this way, we don’t need to modify the actual system, meaning, we will not be mounting system in read-write mode, read-only is fine by us

9- Explore Magisk

So let us start by checking what version of MAGISK is installed on the phone, and where it lives

> adb shell su -c 'magisk -v'
26.4:MAGISK:R
> adb shell su -c 'magisk --path'
/debug_ramdisk
> adb shell su -c 'ls -la /data/adb/modules'
total 24
drwxr-xr-x 3 root root 4096 2026-08-26 12:05 .
drwx------ 6 root root 4096 2026-08-26 12:11 ..
drwxr-xr-x 3 root root 4096 2026-08-26 12:07 hosts

Since this is rootAVD/Magisk, the clean approach is to use a Magisk overlay so we don’t modify the underlying system image.

10- Create the Magisk module with the certificate in it !

we can create our own small Magisk module specifically for the CA without touching the original system image. (the only existing module is hosts)

To create a magisk module we need to do the following

10.1- Create an identical folder structure on my debian machine *(Yours can be windows)

mkdir -p ~/.mitmproxy/module/system/etc/security/cacerts

Move your certificate (xxxxx.0) to the cacerts folder we created above

Now create the module file (Module dfinition file), obviously you can create it with a text editor as well if you wish.

cat > ~/.mitmproxy/module/module.prop <<'EOF'
id=android-mitm-ca
name=Android MITM Analysis CA
version=1.0
versionCode=1
author=Voodoo
description=Custom CA certificate for controlled Android traffic analysis
EOF

Now, put the contents of the module directory (Without the module directory itself) in a zip file

cd ~/.mitmproxy/module
zip -r ../android-mitm-ca.zip .

Now, open the zip file in fileroller (or 7-zip or winrar, whatever suits you) and make sure we have

module.prop
system/
system/etc/
system/etc/security/
system/etc/security/cacerts/
system/etc/security/cacerts/XXXXXXXX.0

If all looks good, super, we have the zip file for android…. go ahead and push it to the android virtual machine

> adb push ~/.mitmproxy/android-mitm-ca.zip /data/local/tmp/

Now, when the above works, install the zip file with magisk

> adb shell su -c 'magisk --install-module /data/local/tmp/android-mitm-ca.zip'
- Device is system-as-root
**************************
 Android MITM Analysis CA 
 by Voodoo 
**************************
*******************
 Powered by Magisk 
*******************
- Extracting module files
- Done

If you get an error above DO NOT REBOOT, only reboot if you get a message like the one i got. if not, Sorry man, you need to reverse the steps above ! something with the rootAVD restore command should do

Now, after rebooting (adb shell reboot), run this to see if our certificate appears !

adb shell su -c 'ls -l /system/etc/security/cacerts/'

Looks fine to me, but let us double check (replace the xxx with your actual file name)

adb shell su -c “ls -l /system/etc/security/cacerts/xxxxxx.0”
-rw-r–r– 1 root root 2000 2026-08-27 20:28 /system/etc/security/cacerts/xxxx.0

And if everything was done correctly, “adb shell su -c ‘mount | grep ” / “‘” should still show read-only

Now, let us play, I am very skeptical that this would work (Unless your android has openssl), but it can’t hurt to try… anyway, failiure here means nothing is the error is that “openssl: inaccessible or not found”, it is expected, and means nothing

adb shell su -c "openssl x509 -in /system/etc/security/cacerts/fea8a918.0 -noout -subject -issuer"

Pinned certificates

there are ways to get around this, ways from things as simple as using apk-mitm to remove the pinning, all the way to more serious stuff such as using jadx to modify the software before playing this game.

Here are some tools that can come in handy for this

• FRIDA: Lets you inject snippets of JavaScript or your own library into native apps + simple tools built on top of the Frida API.
• APK-MITM: (https://github.com/shroudedcode/apk-mitm): Remove certificate pinning from APK files
• objection is a runtime mobile exploration toolkit powered by Frida, which supports certificate pinning bypasses on iOS and Android.
• ssl-kill-switch2 blackbox tool to disable certificate pinning within iOS and macOS applications.
• android-unpinner : From the same people who make the MITMPROXY, it modifies Android APKs to inject Frida and HTTP Toolkit’s unpinning scripts.

apk-mitm

node --version
npm --version
sudo apt install zip unzip
npm install -g apk-mitm

Now, to remove the pinning in your app

apk-mitm <path-to-apk>

The Agentic AI IDE !

Here are a couple official videos

Now, to install google antigravity to debian 12/13

sudo mkdir -p /etc/apt/keyrings
curl -fsSL https://us-central1-apt.pkg.dev/doc/repo-signing-key.gpg | sudo gpg --dearmor -o /etc/apt/keyrings/antigravity-repo-key.gpg

echo "deb [signed-by=/etc/apt/keyrings/antigravity-repo-key.gpg] https://us-central1-apt.pkg.dev/projects/antigravity-auto-updater-dev/ antigravity-debian main" | sudo tee /etc/apt/sources.list.d/antigravity.list > /dev/null

sudo apt update
sudo apt install antigravity

In a previous post, I explained how to install the third party (socialite-ui) on your website, which itself was written to help people with the “Gradually migrating to Laravel” post, but it seems it is not enough, so here is how to fully get socialite working, much of the work is outside your website though, and i won’t be covering that, all you need to do is ask google, for example, how do i enable social logins with google or facebook etc…

The redirect problem

After registering or logging in, your user will be redirected to an absolutely empty “/dashboard“, which is already handled by myrouter.php

Email verification

Before doing anything, let us check that email is working using your SMTP credentials

Create the following route for testing

use Illuminate\Support\Facades\Mail;

Route::get('/test-email', function () {
    Mail::raw('This is a test email.', function ($message) {
        $message->to('test@example.com')
                ->subject('Test Email');
    });
    return 'Email sent!';
});

socialite-ui does not come with email verification (during registration) by default, you will need to enable it, this is simply by following the instructions here

now edit the file (/config/socialite-ui.php) to enable the social logins you want enabled, and alter the other settings as per the page her (https://github.com/Laravel-Uis/socialite-ui)

php artisan config:clear
php artisan cache:clear

Normally this would not have been worthy of a blog post of its own, there are plenty of posts that explain how to do this, but because I don’t want the post on “Gradually migrating to laravel” to be too long, I have decided to move the part on how to setup a vanilla install of laravel to its own post.

Rather than having you setup sociallite in laravel, there is an amazing starter kit on github (https://github.com/Laravel-Uis/socialite-ui) that we will be using to hit the ground running, this is not an official starter kit, it is by a third party, namely Joel Butcher, the same person behind the good old, now unmaintained socialstream (Socialite and JetStream)

So, let’s get down to business, I assume you have a debian machine up and running

apt install php-fpm nginx libnginx-mod-http-headers-more-filter openssl
apt install php-{dev,common,xml,tokenizer,zip,mysql,curl,mbstring,mysql,opcache,gd,intl,json,xsl,bcmath,imap,soap,readline,sqlite3,gmp,guzzlehttp-guzzle}

Now, make sure the following PHP extensions are enabled (i am pretty sure they are after the commands above) but to be safe, check in php.ini

extension=fileinfo
extension=mbstring
extension=openssl

Remember to “mariadb-secure-installation”, then create a username and database in mysql for laravel ! you can either use PHPMYADMIN or from the command line, makes no difference, remember to flush privileges

Install composer and nodejs

apt install composer nodejs npm
composer --version

Now, I am assuming your web directory is owned by the web server user ! first, install laravel globally, this is a new tool we will use to create laravel installations from now on

su - webdev (Whatever user you want to execute under)
composer global require laravel/installer

Now, add laravel to your path in .bashrc

export PATH="$HOME/.config/composer/vendor/bin:$PATH"
source ~/.bashrc
echo $PATH

IMPORTANT: In my case, all my websites are in folders that follow the patern “/var/www/vhosts/com.websitename.www/public”… hence, i execute the lines below from within the “/var/www/vhosts/” directory ! once i execute the below line, i will have a folder called my-app, I usually rename that to com.websitename.www and there is already a public folder in there, The long and short of this story is, your whole project folder with the public directory inside is creating from the command below, if you do it wrong, you can simply move the files and folders in that folder to where they should be

And now, in your project directory (Which is one level below your public web directory), execute the following, the laravel command is the new way, and if you are not looking for Livewire Volt, just visit the github page to see your 3 other options (Vue, React, and Livewire without volt)

laravel new my-app --using=laravel-uis/socialite-ui-livewire-starter-kit

During the install, it will ask you what testing framework you want, You can pick whatever one you prefer, then it will ask you if you would like to run “npm install and npm run build?”, You can hit no, and run the following command yourself

npm install && npm run build

Configure

Now, edit your .env ! the default one uses sqlite, if you want to use mysql / mariadb (like most PHP people), just make sure this becomes your database in your .env file

DB_CONNECTION=mysql
DB_HOST=127.0.0.1
DB_PORT=3306
DB_DATABASE=laravel
DB_USERNAME=root
DB_PASSWORD=

Now, run the following commands

php artisan migrate

The following commands are for reference only

php artisan session:table
php artisan queue:table
php artisan cache:table
php artisan migrate
php artisan config:clear
php artisan route:clear
php artisan view:clear
php artisan cache:clear
php artisan migrate:status

Behind Proxy

If you are behind a proxy, you might need to tell laravel 12 to trust proxies… (Edit /bootstrap/app.php), and add the parts

first, add this line at the top

use Illuminate\Http\Request;

$middleware->trustProxies(
            //at: '*',  // Trust all proxies (use for dynamic/cloud setups like Nginx on the same server; see security notes below)
            at: ['127.0.0.1', '192.168.1.0/24', 'xxx.xxx.xxx.xxx'],
            headers: Request::HEADER_X_FORWARDED_FOR |
                     Request::HEADER_X_FORWARDED_HOST |
                     Request::HEADER_X_FORWARDED_PORT |
                     Request::HEADER_X_FORWARDED_PROTO |
                     Request::HEADER_X_FORWARDED_AWS_ELB  // Include if using AWS; optional for standard Nginx
        );

Right where there is an empt comment inside

->withMiddleware(function (Middleware $middleware) {

This looks long, but it’s simpler than it seems.

Scenario: You have an existing legacy PHP system, and you would like to gradually migrate to Laravel, you don’t want to migrate everything to laravel from the word go (because you either don’t have the time, or you need features from laravel but don’t want to learn laravel all at once, or maybe developing with laravel is your favorite way of learning laravel), whichever it may be, this tutorial is for you.

To get on track, I will assume that all you want for the time being is to have laravel (with socialite) take care of registering and authenticating users, and pass the logged in user data to your system… and leave other migrations for later, this is a good way to get started.

The steps

1- Install Laravel 12

Just a plain old vanilla setup. If you don’t know how to do this, I have the following post for you (Installing laravel 12 with social login), you don’t need to modify anything for this setup to work, yet you might find some stuff that needs some modification (Regardless of this setup), so here are a few modifications you may want to make

all you need now is to make sure you can register as a user, either directly or using social login (Or whatever you want), to speed things up, you might want to use this amazing starter kit here (https://github.com/Laravel-Uis/socialite-ui), bottom line is  a clean Laravel install — nothing fancy..

2- One of the following two options depending on how your system is setup

Option 1– If your system already uses rewrite, have nginx rewrite all your requests to myrouter.php instead of index.php, here is the nginx config file I use, and here is the myrouter.php file

Elaboration: Normally, in a Laravel installation, nginx would rewrite all requests to laravel’s own index.php, what we have done with those two files is have it rewrite it to our own router (myrouter.php), if our existing system knows how to handle the request, it would, if it doesn’t , then it is probably a laravel page that the user is seeking (such as registration page), so we pass control to laravel.

Option 2– If you are not using rewrite in your legacy system, and PHP pages are accessed directly, the laravel recommended nginx config already deals with this since it checks for file existence before rewriting the request to index.php, all you will need to do at this stage is include this small file in your PHP files (require_once rather than include)

3- Integrating the auth system into ours

Our options: What we will do is create an endpoint on Laravel and have our system call it internally

What we won’t do: it is worth mentioning that there are quite a few ways besides the above, but none of them is straight forward, one of them is to include laravel’s internals (the console bootstrap path… what Artisan uses) into our own system, but Laravel’s sessions are serialized and encrypted ( with Illuminate\Encryption\Encrypter and the APP_KEY) so you will have to replicate Laravel’s decryption/unserialization logic,so we will stick to the auth endpoint… and optimize it by only firing it up when needed (To not load laravel for every walk in visitor to the website)

Note: If you will be going by my advice to develop them separately, what comes below needs laravel to function/test/develop, you will need to create a file that checks if laravel is available before firing up, and if not, send back a dummy testing user (or whatever you need), you can make that easier by toggling a “production” flag, and when in production it calls upon laravel, when in development, it serves the dummy user.

3.1 : Create an endpoint in laravel, very simple, Just edit laravel’s routes file ( routes/web.php) and add the following, it is also wrap it in middleware that only allows access from localhost, even though that is not really needed, but keeping security tight is a good idea (You can remove the extra wrapper by deleting everything from ->middleware through the end and putting a semi colon instead, up to you)

Note: I personally added this to /routes/auth.php alongside the login and registration features since it is authentication specific, but technically it makes absolutely no difference

use Illuminate\Support\Facades\Route;

Route::get('/auth/me', function () {
    return response()->json([
        'user' => auth()->check() ? auth()->user() : null,
    ]);
})->middleware(function ($request, $next) {
    if ($request->server->get('SERVER_ADDR') !== $request->ip()) {
        abort(403);
    }
    return $next($request);
});

Now, we need a function to talk to laravel internally and get the user ! we will fire up laravel, take the user data, and go back to our own system

And we also need to know when to call that function, there is no need to fire up Laravel then tear it all down when there is no chance that this might be a logged in user !

Previously, i had a function that fired up laravel internally, but after some digging, I realized that the overhead of calling it over http is very low, and that it will future proof the system (When laravel decide to change things)

function currentUser($http_port) {
    static $cachedUser = null;
    static $resolved = false;

    if (!$resolved) {
        $cachedUser = null; // default

        if (!empty($_COOKIE['voodoo_session'])) {
            //$ch = curl_init("https://www.voodoo.business/auth/me"); // Using your FQDN
            $ch = curl_init("http://127.0.0.1:{$http_port}/auth/me");
            curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
            curl_setopt($ch, CURLOPT_HEADER, false);
            curl_setopt($ch, CURLOPT_TIMEOUT, 5); // optional timeout
            curl_setopt($ch, CURLOPT_HTTPHEADER, [
                'Host: www.voodoo.business',
                'Cookie: voodoo_session=' . $_COOKIE['voodoo_session']
            ]);

            $response = curl_exec($ch);

            if ($response === false) {
                error_log('currentUser cURL error: ' . curl_error($ch));
            } else {
                $data = json_decode($response, true);

                if (json_last_error() !== JSON_ERROR_NONE) {
                    error_log('currentUser JSON decode error: ' . json_last_error_msg());
                } else {
                    $cachedUser = $data['user'] ?? null;
                }
            }

            curl_close($ch);
        }

        $resolved = true;
    }

    return $cachedUser;
}

Now you can simply call the function currentUser as follows, as many times as you wish, it will only invoke laravel once per web request

$user = currentUser();
if ($user) {
    echo "Logged in as {$user['name']}";
} else {
    echo "Not logged in";
}

Improvement – Optimization: You can short circuit earlier by simply connecting to the database and checking if the session id ($sessionId = $_COOKIE[‘laravel_session’];) exists in the database or the file or memory store, then, if it exists, invoke the currentUser function… this will ( check that the session file/row/key exists, not that it’s valid), but getting into this now will complicate the tutorial because there are 3 different scenarios and the solution we are at right now is good enough for most low to medium traffic websites

The logout button

Laravel does not have a page that logs you out as soon as you visit it, this will effectively disable CSRF ! and allow forced logout, so the answer is to create a logout page in Laravel and visit that page, it is very easy

The route file

/routes/web.php

Route::get('/log_me_out', function () {
    return view('logout'); // blade template with the button
});

The Blade template

resources/views/logout.blade.php (If you are not familiar with laravel, the part of the file name that is before “.blade.php” is the view name, so the view in the route takes us here

<h1>Click below to log out</h1>

<form method="POST" action="{{ route('logout') }}">
    @csrf
    <button type="submit">Logout</button>
</form>

All you need to do now is link to /log_me_out, and the user will be presented with the option to logout

Thoughts and ideas

I do not practice what I preach, my projects live inside of each other, the outer project is laravel, and my project lives inside a folder inside laravel (Not inside the public folder, just inside laravel’s project folder), at which point, myrouter.php either reads static files and serves them, or includes php files from my projects folder, so I only have 1 file inside the public folder….

So, in my projects folder, i do the regular steps below

cd /var/www/laravel/innderproject (This is the whole story, running composer inside the project folder)
composer init
composer require nikic/php-parser:^5.6 vlucas/phpdotenv:^5.6

And inside that folder, all files include the code

require __DIR__ . '/vendor/autoload.php';

This loads from the inner vendors folder, laravel knows how to reach the outer one

Error: Unable to locate package php8.4-imap
Error: Couldn’t find any package by glob ‘php8.4-imap’

the debian package maintainer (Ondřej Surý) has released it on his repository, but it is nowhere to be found for Debian 13 on “https://packages.debian.org/“, trixie didn’t trickle down to the website yet, but in SID, it reads “The extension is unbundled from php-src as of PHP 8.4.0.”

I am not yet sure what this means, there is a chance it means we should install it using PECL, I will not do that, I would rather installwith apt, but if i wanted to, i would execute

sudo apt install php-pear php8.4-dev
sudo pecl install imap

There is a chance that it will be coming back to debian’s repositories since Sury (The package maintainer) has it, but I have no idea when

So, let us install it (the IMAP extension and nothing else) from the sury repository to make things simpler when debian has it again

Let us start by adding

sudo apt install -y lsb-release apt-transport-https ca-certificates curl gnupg2
echo "deb https://packages.sury.org/php $(lsb_release -sc) main" | sudo tee /etc/apt/sources.list.d/sury-php.list
curl -fsSL https://packages.sury.org/php/apt.gpg | sudo gpg --dearmor -o /etc/apt/trusted.gpg.d/sury-php.gpg
sudo apt update

Now we have added Sury’s repository to apt, let us go ahead and pin it so that only php8.4-imap is sourced from that repo !

sudo tee /etc/apt/preferences.d/sury-php.pref >/dev/null <<'EOF'
Package: *
Pin: release o=deb.sury.org
Pin-Priority: 1

Package: php8.4-imap
Pin: release o=deb.sury.org
Pin-Priority: 990
EOF

sudo apt update

Now install

sudo apt install php8.4-imap

To make sure everything is correct

apt policy php8.4-imap

Now in my case, I need to restart PHP (systemctl restart php8.4), in your case, you might need to restart apache or something else, when in doubt, restart the machine 😛

Once the package is in debian repositories again, we will simply remove /etc/apt/preferences.d/sury-php.pref and /etc/apt/sources.list.d/sury-php.list, then reinstall php8.4-imap.

BeautifulSoup is a python package that allows you to extract data from HTML files, it is very easy and intuitive

Let us assume you have an HTML page !

First, let us assume you want the title from that HTML page….

mysoup = BeautifulSoup(response.content, 'html.parser')
title = soup.title.string if soup.title else "No title found"

Now, assuming you want to remove everything that has to do with CSS and presentation, you can remove the following things with this easy code snippet, then putting whatever is lef in a variable called text

for irrelevant in mysoup.body(["script", "style", "img", "input"]):
            irrelevant.decompose()
text = soup.body.get_text(separator="\n", strip=True)

If you are familiar with python, you probably are also familiar with pip, and very likely familiar with venv (Virtual environments)

There is nothing special in particular about this for AI, it is exactly as you would for any other installation.

For AI, i would totally recommend anaconda, but if for some reason that is not an option, this option will do 100% of the time

So, you need to start by installing Python 3 !, On debian, I would just use the repository with apt, it may be true at the time of writing that in the repo it uses 3.11 rather than the latest 3.13, but that is absolutely fine

sudo apt update
// The following command should do
sudo apt install python3
//But i would much rather install everything python in one go
apt install build-essential wget git python3-pip python3-dev python3-venv \
     python3-wheel libfreetype6-dev libxml2-dev libzip-dev libsasl2-dev \
     python3-setuptools

Now, with that out of the way, navigate to the project’s folder (Assuming you have downloaded a project for example), and create a virtual environment

python3 -m venv venv

Now, you can activate that project with

source venv/bin/activate
//On windows, the above should look something like
venv\Scripts\activate

That is basically it, you will now need to, from within the command prompt of the venv, install dependencies either one by one using the pip command, or point it to a file containing the dependencies, for example

pip install -r requirements.txt

You should now be good to go !

What is Anaconda ?

Conda, Like pip, is a python package manager, but conda is probably undisputed as the more thorough solution of the two, with better support for non-python packages (pip has very limited support) and support for more complex dependency trees

To clarify things, conda is the package manager while Anaconda is a bigger bundle, if you want to install conda alone, you are probably looking to install Miniconda. Anaconda is a set of about a hundred packages including conda, numpy, scipy, ipython notebook, and so on.

So, let us go through installing and using Anaconda on all 3 platforms, Windows, Linux and Mac

Linux

On Debian, there is no Anaconda package, to install, you will need to download the download script from anaconda and install it (Or conda, or miniconda for that matter) , you can add miniconda to apt using the “https://repo.anaconda.com” repo if you are willing to add it (apt install conda), but here I will assume you will just install Anaconda, and the only orthodox way to do that is with the installation script

Download the Anaconda installer from the Anaconda Website (There is a button that reads skip registration if you don’t want to give them your email address)

https://www.anaconda.com/download

Navigate to the downloads folder, and execute the script just downloaded, in my case, the script’s name was Anaconda3-2024.10-1-Linux-x86_64.sh so I execute the following

cd /home/qworqs/Downloads
chmod 0777 Anaconda3-2024.10-1-Linux-x86_64.sh
./Anaconda3-2024.10-1-Linux-x86_64.sh

After accepting the agreement, I see a message asking me to accept the license agreement, hit enter, take a look at the license agreement, hit the letter (q) to exit the agreement, then you will be asked if you accept the agreement, assuming you agreed to it, you will next be presented with….

Anaconda3 will now be installed into this location:
/home/qworqs/anaconda3

To which i accepted the suggested location

Now, I opt to keep the installer in the downloads directory just in case something goes wrong, but you can safely delete the 1GB installer if you like !

At the end of the installation, the installer offers to update your shell, in my case, i opted NOT TO, if you opted otherwise, you can always “set auto_activate_base” to false….

Do you wish to update your shell profile to automatically initialize conda?
This will activate conda on startup and change the command prompt when activated.
If you'd prefer that conda's base environment not be activated on startup,
   run the following command when conda is activated:

conda config --set auto_activate_base false

You can undo this by running `conda init --reverse $SHELL`? [yes|no]

Once i answered no, i was presented with the following message

You have chosen to not have conda modify your shell scripts at all.
To activate conda's base environment in your current shell session:

eval "$(/home/voodoo/anaconda3/bin/conda shell.YOUR_SHELL_NAME hook)" 

To install conda's shell functions for easier access, first activate, then:

conda init

Thank you for installing Anaconda3!

The environment

For convinience, let us start by adding our Anaconda3 installation to system path

//First, add anaconda to path by editing either ./bashrc OR ~/.bash_profile (Depending on which one you have), and adding the following to the bottom of the file

export PATH=~/anaconda3/bin:$PATH

Now, to apply the changes, you should either close the terminal window and re-open it, or run the command “source ~/.bashrc” or “source ~/.bash_profile”

To check whether the magic happened, run the command “conda –version“, in my case, that returned “conda 24.9.2”

Now, from this stage on, conda is installed, but to be able to use it, you should have a project ! so now you can move on (The index page), and I will explain how to run your project where needed, here, for completion, I will assume you have a project and put the instructions here (You know you are in the project dir when you see a yaml file commonly called environment.yml)

So, again, to activate an environment, there is a yaml file for every project that contains the dependencies of that project ! let us assume you are in your project’s directory, and the yaml file is called environment.yml , the following command will create a python sub-environment, and install all the dependencies in that yaml file, be sure you are in the directory with the yaml file

Now, to create a virtual environment, cd into the directory that has your project and run the following

conda env create -f environment.yml

Once the above is done downloading and installing, you should get a message like the one below

#
# To activate this environment, use
#
#     $ conda activate projectName
#
# To deactivate an active environment, use
#
#     $ conda deactivate

Now, next time, when you open a terminal, and want to activate an environment

1- conda init (conda deactivate to reverse), you only need to do this once
2- open a new shell
3- conda activate ProjectName (Also, conda deactivate to exit)

I recommend you read this after reading My Dart Notes.

this document is a reminder of everything Dart related ! There is a different one for flutter.

Why ? Because I have a problem, I don’t limit myself to a few languages like normal human beings, I know A LOT of programming languages, and when you don’t use a language long enough, you start messing up which syntax works where, so I create those small “Reminder” pages that I can skim through in an hour before I start working with any language after leaving it for some time !

Mind you, I eventually lose the page “in the (It is on some hard drive somewhere) and that is because after using it a couple of times, I no longer need it. but when i come across old ones, I will post them on this blog.

I am posting this one online because I am composing it now, and I haven’t lost it yet

Continue reading “My Flutter notes”