diff options
70 files changed, 1614 insertions, 2878 deletions
diff --git a/INTERESTING_PROJECTS.org b/INTERESTING_PROJECTS.org index 22eaba70..109a4f0a 100644 --- a/INTERESTING_PROJECTS.org +++ b/INTERESTING_PROJECTS.org @@ -308,6 +308,7 @@ See also [[TOOLS.org][tools I use]]. * Hardware - https://junkerhq.net/xrgb/index.php/OSSC - The Open Source Scan Converter is a low-latency video digitizer and scan converter designed primarily for connecting retro video game consoles and home computers to modern displays. - https://openterface.com/ - Adds KVM capabilities to a laptop. + - https://www.dpin.de/nf/linux-music-making-sonic-pi-lmms-akai-mpk-usb-midi/ has software recommendations and instructions to adjust velocity on an AKAI MPK Mini ** Phones - https://github.com/Dakkaron/Fairberry - add Blackberry keyboard to other phones * Technical writing @@ -10,7 +10,7 @@ Como nunca tengo oportunidad de trabajar en modo monorepo, hay que aprovechar es * [Gran parte del código que soporta mi infraestructura personal](personal_infra/). * [Ideas en las que me gustaría trabajar algún día, pero que cuelgo aquí por si algún incauto pica](IDEAS.org). * [Herramientas que uso](TOOLS.org). -* Artículos varios un poco tipo wiki sobre diversos temas, como [hardware](hardware/), [Linux](linux/) y [programación](programming/). +* Artículos varios un poco tipo wiki sobre diversos temas. * [Mi configuración de Emacs](emacs/). * [Unos cuantos scripts que uso en mis "estaciones de trabajo"](scripts/) y [un par de barbaridades para montar un contenedor para Distrobox/Toolbox con algunas de las herramientas que uso](workstation/). * [La definición del modelo de datos del sistema que uso para registrar métricas de salud](weight/), que se convierte en una web gracias a [zqxjkcrud](https://github.com/alexpdp7/zqxjkcrud/), mi framework para desarrollo rápido de CRUD. @@ -30,7 +30,7 @@ Más cosas escritas [en español](https://alex.corcoles.net/notas/) y [en inglé * [Most of the code for my personal infra](personal_infra/). * [Ideas I'd like to work on someday, but mostly making them public to bait people into implementing them instead of me](IDEAS.org). * [Tools I use](TOOLS.org). -* Wiki-like articles on topics such as [hardware](hardware/), [Linux](linux/), and [programming](programming/). +* Wiki-like articles. * [My emacs config](emacs/). * [Scripts I use on my workstations](scripts/) and [some bizarre stuff to build Distrobox/Toolbox containers with some tools I use](workstation/). * [The database schema for my health metrics tracking system](weight/), which has a web UI via [zqxjkcrud](https://github.com/alexpdp7/zqxjkcrud/), my CRUD rapid development framework. diff --git a/blog/content/2014/01/integracion-apis-y-no-me-toques-los-bezos.gmi b/blog/content/2014/01/integracion-apis-y-no-me-toques-los-bezos.gmi index 4cc8a2fb..070e3d7d 100644 --- a/blog/content/2014/01/integracion-apis-y-no-me-toques-los-bezos.gmi +++ b/blog/content/2014/01/integracion-apis-y-no-me-toques-los-bezos.gmi @@ -2,23 +2,18 @@ Según Steve Yegge, un día Jeff Bezos, el capo de Amazon, envió un memorándum interno que venía a decir: -> 1. -> All teams will henceforth expose their data and functionality through service interfaces. -> -> 2. -> Teams must communicate with each other through these interfaces. -> -> 3. -> There will be no other form of interprocess communication allowed: no direct linking, no direct reads of another team's data store, no shared-memory model, no back-doors whatsoever. The only communication allowed is via service interface calls over the network. -> -> 4. -> It doesn't matter what technology they use. HTTP, Corba, Pubsub, custom protocols -- doesn't matter. Bezos doesn't care. -> -> 5. -> All service interfaces, without exception, must be designed from the ground up to be externalizable. That is to say, the team must plan and design to be able to expose the interface to developers in the outside world. No exceptions. -> -> 6. -> Anyone who doesn't do this will be fired. +> 1. All teams will henceforth expose their data and functionality through service interfaces. + +> 2.Teams must communicate with each other through these interfaces. + +> 3. There will be no other form of interprocess communication allowed: no direct linking, no direct reads of another team's data store, no shared-memory model, no back-doors whatsoever. The only communication allowed is via service interface calls over the network. + +> 4. It doesn't matter what technology they use. HTTP, Corba, Pubsub, custom protocols -- doesn't matter. Bezos doesn't care. + +> 5. All service interfaces, without exception, must be designed from the ground up to be externalizable. That is to say, the team must plan and design to be able to expose the interface to developers in the outside world. No exceptions. + +> 6. Anyone who doesn't do this will be fired. + Este puñetero memo, y toda la gente que lo ha leído y ha aplicado el silogismo falaz de "En Amazon son unos cracks, en Amazon siguen este credo ergo si yo sigo este credo seré un crack" son un maldito dolor de cabeza. Conste que no es que considere que las APIs sean una plaga a exterminar, ni mucho menos, pero debería olernos mal las sentencias absolutistas y los razonamientos de talla única. No todos los entornos son iguales, ni se enfrentan a los mismos problemas- y por tanto lo que funciona para unos, no funciona para otros. Adicionalmente, es difícil razonar que la aplicación ciega de esta doctrina es lo que ha llevado a Amazon al éxito y estoy seguro que no en pocas ocasiones mejor les hubiera ido siendo un poco más críticos. diff --git a/blog/content/2026/03/notas.gmi b/blog/content/2026/03/notas.gmi new file mode 100644 index 00000000..28e188a7 --- /dev/null +++ b/blog/content/2026/03/notas.gmi @@ -0,0 +1,32 @@ +# 2026-03-01 Notas + +Hoy tocaba escribir en el blog, pero en vez de ello me he dedicado a migrar más contenido (principalmente "ensayos") a esta web. + +Durante mucho tiempo, metía ficheros de texto (mayoritariamente en Markdown) en mi "monorepo" Git personal. (El monorepo es un amalgama de un montón de cosas, incluido este blog y muchas más cosas.) Esta técnica no funciona mal y es muy cómoda, pero me ataba demasiado a GitHub. + +Ahora podéis leer ese contenido en: + +=> ../../notes Notes (en inglés) +=> ../../notas Notas (en español) + +Con lo que ahora todo tiene URLs controladas por un servidor. + +Además, he convertido todo (a manopla) al formato gemtext de Gemini. gemtext es mucho más limitado que Markdown, con lo que he tenido que simplificar un poco el formato. Pero creo que esta "sencillez obligada" me ayuda a estructurar las cosas de una manera más simple y más adaptable a distintos formatos. (Como que por ejemplo, podéis leer todo usando un navegador de Gemini, lo que ofrece en mi opinión muchas ventajas.) + +Tras borrar también bastante contenido obsoleto o de baja calidad, creo que ya no queda casi nada susceptible de pasarse a gemtext. Todavía hay algo material, pero es mayormente: + +* Listados en formato Org con mucha jerarquía que no funcionarían muy bien en gemtext. +* Un par de artículos que debería actualizar y modernizar bastante... cosa que seguramente no haré a corto plazo y esperaré a que necesiten un remodelado profundo. + +Con esto creo que cierro una primera fase de la iteración de este sitio web (ahora cápsula y no sólo blog). Curiosamente he "perdido" funcionalidad: + +* Falta navegabilidad; básicamente sólo podemos desandar nuestros pasos en la nueva web usando el botón de "atrás" de nuestro navegador. +* He perdido la editabilidad que me daba GitHub; antes los textos en GitHub tenían un botón editar la mar de majo y las entradas del blog tenían un enlace para editarlas en GitHub. + +Lo primero creo que tengo maneras de resolverlas que intentaré aplicar igual a corto plazo. + +Lo segundo me fastidia mucho más y no se me ocurre una buena manera de subsanarlo. Podría seguir usando las funcionalidades de GitHub, pero claro, estoy intentando reducir mi dependencia. Podría usar software equivalente a GitHub, pero me estoy inclinando por alternativas mucho más sencillas para hospedar Git que no tienen funcionalidad equivalente (y que me resultan muy atractivas por otros motivos). + +Para cerrar comentaré que ahora mismo el contenido de esta web es más o menos de 700.000 palabras. A veces en mi cabeza le llamo a esto "mi universo cinemático", porque no es más que una película que me he montado en la cabeza durante más de un cuarto de siglo. Por suerte o por desgracia no soy Jack Kerouac[1], pero al menos os diré que tanta verborrea me entretiene y creo que me enriquece. Os animo a montar vuestros propios universos cinemáticos. + +=> https://sabr.org/journal/article/jack-kerouac-the-beat-of-fantasy-baseball/ [1] Jack Kerouac: The Beat of Fantasy Baseball diff --git a/blog/content/notas/ficcion/cuentos_del_triangulo_verde.gmi b/blog/content/notas/ficcion/cuentos-del-triangulo-verde.gmi index 3d37bfaa..3d37bfaa 100644 --- a/blog/content/notas/ficcion/cuentos_del_triangulo_verde.gmi +++ b/blog/content/notas/ficcion/cuentos-del-triangulo-verde.gmi diff --git a/blog/content/notas/ficcion/el_principe.gmi b/blog/content/notas/ficcion/el-principe.gmi index f1db64a9..f1db64a9 100644 --- a/blog/content/notas/ficcion/el_principe.gmi +++ b/blog/content/notas/ficcion/el-principe.gmi diff --git a/blog/content/notas/ficcion/en_los_mejores_cines.gmi b/blog/content/notas/ficcion/en-los-mejores-cines.gmi index dcb32f3a..dcb32f3a 100644 --- a/blog/content/notas/ficcion/en_los_mejores_cines.gmi +++ b/blog/content/notas/ficcion/en-los-mejores-cines.gmi diff --git a/blog/content/notas/ficcion/invoco_al_diablo.gmi b/blog/content/notas/ficcion/invoco-al-diablo.gmi index 3427bc57..3427bc57 100644 --- a/blog/content/notas/ficcion/invoco_al_diablo.gmi +++ b/blog/content/notas/ficcion/invoco-al-diablo.gmi diff --git a/blog/content/notas/ficcion/lucifer_martinez.gmi b/blog/content/notas/ficcion/lucifer-martinez.gmi index d410f065..d410f065 100644 --- a/blog/content/notas/ficcion/lucifer_martinez.gmi +++ b/blog/content/notas/ficcion/lucifer-martinez.gmi diff --git a/blog/content/notas/ficcion/maldito_clip.gmi b/blog/content/notas/ficcion/maldito-clip.gmi index 648ef923..648ef923 100644 --- a/blog/content/notas/ficcion/maldito_clip.gmi +++ b/blog/content/notas/ficcion/maldito-clip.gmi diff --git a/blog/content/notas/ficcion/mariano_el_programador.gmi b/blog/content/notas/ficcion/mariano-el-programador.gmi index f6eff640..f6eff640 100644 --- a/blog/content/notas/ficcion/mariano_el_programador.gmi +++ b/blog/content/notas/ficcion/mariano-el-programador.gmi diff --git a/blog/content/notas/ficcion/un_paseo_por_el_rio.gmi b/blog/content/notas/ficcion/un-paseo-por-el-rio.gmi index 611fd599..611fd599 100644 --- a/blog/content/notas/ficcion/un_paseo_por_el_rio.gmi +++ b/blog/content/notas/ficcion/un-paseo-por-el-rio.gmi diff --git a/blog/content/notas/index.gmi b/blog/content/notas/index.gmi index 8908ba55..5c0cc7dc 100644 --- a/blog/content/notas/index.gmi +++ b/blog/content/notas/index.gmi @@ -6,18 +6,24 @@ ## Ficción -=> ficcion/cuentos_del_triangulo_verde Cuentos del triángulo verde -=> ficcion/el_principe El príncipe, el rey y el verdadero mal -=> ficcion/maldito_clip Maldito clip +=> ficcion/cuentos-del-triangulo-verde Cuentos del triángulo verde +=> ficcion/el-principe El príncipe, el rey y el verdadero mal +=> ficcion/maldito-clip Maldito clip -=> ficcion/lucifer_martinez Lucifer Martínez +=> ficcion/lucifer-martinez Lucifer Martínez => ficcion/conmutatividad Conmutatividad -=> ficcion/mariano_el_programador Mariano, el programador -=> ficcion/en_los_mejores_cines En los mejores cines -=> ficcion/un_paseo_por_el_rio Un paseo por el río -=> ficcion/invoco_al_diablo Invocó al diablo y lo que sucedió a continuación te sorprenderá +=> ficcion/mariano-el-programador Mariano, el programador +=> ficcion/en-los-mejores-cines En los mejores cines +=> ficcion/un-paseo-por-el-rio Un paseo por el río +=> ficcion/invoco-al-diablo Invocó al diablo y lo que sucedió a continuación te sorprenderá ## Tele => tele/el-estado-de-esta-nacion-grabando El estado de esta nación grabando => tele/teles Teles + +## Tecnología + +=> tecnologia/quiero-instalar-linux Quiero instalar Linux +=> tecnologia/problemas Problemas +=> tecnologia/mama-quiero-ser-programador Mamá, quiero ser programador diff --git a/programming/mama_quiero_ser_programador.md b/blog/content/notas/tecnologia/mama-quiero-ser-programador.gmi index 7fc8ded3..01388fe2 100644 --- a/programming/mama_quiero_ser_programador.md +++ b/blog/content/notas/tecnologia/mama-quiero-ser-programador.gmi @@ -1,8 +1,6 @@ # Mamá, quiero ser programador -Nuestro primer ordenador llego a casa cuando yo tenía cuatro años. -A mi madre le gusta repetir que aquel día mi padre, mi hermano y yo no comimos. -Desde entonces me han fascinado los ordenadores, lo que me ha llevado a la fascinación por programar. +Nuestro primer ordenador llego a casa cuando yo tenía cuatro años. A mi madre le gusta repetir que aquel día mi padre, mi hermano y yo no comimos. Desde entonces me han fascinado los ordenadores, lo que me ha llevado a la fascinación por programar. Curiosamente, estudié ingeniería informática un poco por casualidad e incluso cuando acabé la carrera, tenía mis dudas de si sería mi carrera profesional. @@ -14,26 +12,19 @@ Este texto intenta recoger mis opiniones sobre estos temas. ## Observaciones sobre el mercado laboral -Los trabajos de programador parecen reflejar que hay mucho trabajo y poca gente capacitada para hacerlo. -Hay bastantes trabajos comparativamente bien pagados y con buenas condiciones. +Los trabajos de programador parecen reflejar que hay mucho trabajo y poca gente capacitada para hacerlo. Hay bastantes trabajos comparativamente bien pagados y con buenas condiciones. -Sin embargo, esto mayormente aplica a los trabajadores con bastante experiencia. -Gente con poca experiencia comenta que encontrar un trabajo requiere un esfuerzo desproporcionado, si se consigue encontrar siquiera trabajo. +Sin embargo, esto mayormente aplica a los trabajadores con bastante experiencia. Gente con poca experiencia comenta que encontrar un trabajo requiere un esfuerzo desproporcionado, si se consigue encontrar siquiera trabajo. Propongo que esto se debe a que la mayoría de ofertas son para programadores con experiencia y el atractivo de la profesión ha generado un número de candidatos mucho mayor que las ofertas disponibles, creando la situación inversa que la de los programadores con experiencia. -Además, la profesión de programador tiene la peculiaridad de que muchos profesionales dedican mucho de su tiempo libre a ejercer, más allá del trabajo o los estudios. -Como el proceso de obtener un trabajo tiene elementos de competencia con el resto de candidatos a un puesto, entre programadores con poca experiencia se ha popularizado autoformarse para mejorar sus posibilidades. -Esta medida sólo es efectiva cuando nos permite destacar sobre otros competidores, con lo que cada vez parece necesitarse más esfuerzo de autoformación para competir. +Además, la profesión de programador tiene la peculiaridad de que muchos profesionales dedican mucho de su tiempo libre a ejercer, más allá del trabajo o los estudios. Como el proceso de obtener un trabajo tiene elementos de competencia con el resto de candidatos a un puesto, entre programadores con poca experiencia se ha popularizado autoformarse para mejorar sus posibilidades. Esta medida sólo es efectiva cuando nos permite destacar sobre otros competidores, con lo que cada vez parece necesitarse más esfuerzo de autoformación para competir. ### La irrupción de los LLMs y la crisis postpandemia -En 2020, diversos factores generaron un crecimiento del sector mayor de lo habitual. -Sin embargo, a partir de mediados de 2022, se dispararon los despidos en el sector. -<https://layoffs.fyi> recoge cifras de despidos que desde el segundo trimestre de 2022 siempre se han mantenido como mínimo bastante por encima del periodo 2020-2021, con un pico de despidos en el tercer trimestre de 2023. +En 2020, diversos factores generaron un crecimiento del sector mayor de lo habitual. Sin embargo, a partir de mediados de 2022, se dispararon los despidos en el sector. layoffs.fyi recoge cifras de despidos que desde el segundo trimestre de 2022 siempre se han mantenido como mínimo bastante por encima del periodo 2020-2021, con un pico de despidos en el tercer trimestre de 2023. -Además, en noviembre de 2022, OpenAI lanzó ChatGPT. -Desde entonces, muchos han augurado que los LLMs pueden afectar significativamente al mercado laboral en general y al sector en particular. +Además, en noviembre de 2022, OpenAI lanzó ChatGPT. Desde entonces, muchos han augurado que los LLMs pueden afectar significativamente al mercado laboral en general y al sector en particular. Finalmente, muchos interpretan movimientos políticos, económicos y otras inestabilidades como otra crisis mundial en ciernes. @@ -51,42 +42,33 @@ No hay certezas para predecir el futuro, pero podemos observar el pasado. Apostar por la programación antes parece una buena idea a posteriori, pero mucha gente ha abandonado el sector y no todo el mundo ha tenido trabajos buenos y bien pagados. -[Face it: you're a crazy person](https://www.experimental-history.com/p/face-it-youre-a-crazy-person) es un artículo que propone que escoger una profesión debería basarse en lo atractivo que nos resultan *todas* las partes del trabajo, sobre todo las peores. +=> https://www.experimental-history.com/p/face-it-youre-a-crazy-person Face it: you're a crazy person es un artículo que propone que escoger una profesión debería basarse en lo atractivo que nos resultan *todas* las partes del trabajo, sobre todo las peores. Ya cuando estudiaba, mucha gente se imaginaba divirtiéndose programando videojuegos. -En mi opinión, algunas de las peores partes de la programación son las prisas; siempre se hace todo con menos tiempo del que querríamos. -Eso influye en que lo que hacemos y lo que usamos suele estar mal documentado o no funciona bien, haciendo que la programación sea menos "construir cosas que sirven de algo" y más "reparar con mil chapuzas cosas ligeramente estropeadas". +En mi opinión, algunas de las peores partes de la programación son las prisas; siempre se hace todo con menos tiempo del que querríamos. Eso influye en que lo que hacemos y lo que usamos suele estar mal documentado o no funciona bien, haciendo que la programación sea menos "construir cosas que sirven de algo" y más "reparar con mil chapuzas cosas ligeramente estropeadas". -Además, lo otro es que muy probablemente tendremos que dedicar tiempo no remunerado y fuera de nuestra formación en formarnos, en general haciendo cosas que si bien pueden resultar más gratificantes, en general también serán frustrantes. -(Además, de cara a conseguir trabajo, lamentablemente en general también ayudará muchísimo *completar* cosas que podamos poner en nuestro currículum.) +Además, lo otro es que muy probablemente tendremos que dedicar tiempo no remunerado y fuera de nuestra formación en formarnos, en general haciendo cosas que si bien pueden resultar más gratificantes, en general también serán frustrantes. (Además, de cara a conseguir trabajo, lamentablemente en general también ayudará muchísimo *completar* cosas que podamos poner en nuestro currículum.) -A otro nivel, los trabajos de programación que pudieran resultar más motivadores y edificantes son por general los peor pagados y con peores condiciones, mientras que los buenos suelen ser en general los que despertarán menos vocación en nadie. -En mi opinión, es complicado conseguir algo de realización en este sector sin sacrificar la mayoría de beneficios que muchos ven en la profesión. +A otro nivel, los trabajos de programación que pudieran resultar más motivadores y edificantes son por general los peor pagados y con peores condiciones, mientras que los buenos suelen ser en general los que despertarán menos vocación en nadie. En mi opinión, es complicado conseguir algo de realización en este sector sin sacrificar la mayoría de beneficios que muchos ven en la profesión. -A corto plazo, mi previsión es que todo esto empeore. -El sector muy probablemente seguirá siendo una opción mucho mejor que la mayoría, pero creo que las expectativas laborales deberán rebajarse. -El único consejo que se me ocurre es intentar construir cosas similares a las que vemos en el mundo real para ver si nos gusta realmente el trabajo. +A corto plazo, mi previsión es que todo esto empeore. El sector muy probablemente seguirá siendo una opción mucho mejor que la mayoría, pero creo que las expectativas laborales deberán rebajarse. El único consejo que se me ocurre es intentar construir cosas similares a las que vemos en el mundo real para ver si nos gusta realmente el trabajo. ## Consiguiendo un trabajo Los procesos de contratación son una parte proporcionalmente muy pequeña de la vida laboral pero que concentran gran parte de lo que se habla y se protesta en este sector. -Tengo más de dos décadas de experiencia profesional, creo que he tenido muy buenos trabajos y generalmente voy a puestos con menos competencia de lo normal. -Pero para encontrar trabajo, en ocasiones he tenido que presentarme a más de un centenar de ofertas y llevarme innumerables rechazos de todo tipo, silenciosos y sonoros. +Tengo más de dos décadas de experiencia profesional, creo que he tenido muy buenos trabajos y generalmente voy a puestos con menos competencia de lo normal. Pero para encontrar trabajo, en ocasiones he tenido que presentarme a más de un centenar de ofertas y llevarme innumerables rechazos de todo tipo, silenciosos y sonoros. -Hay estudios que parecen demostrar que una parte muy importante de ofertas en el sector incluso son totalmente ficticias. -(Esto seguramente afecte a otros sectores, pero parece especialmente popular en este.) +Hay estudios que parecen demostrar que una parte muy importante de ofertas en el sector incluso son totalmente ficticias. (Esto seguramente afecte a otros sectores, pero parece especialmente popular en este.) Los procesos de selección de personal tienen una gran parte de competencia porque en general, siempre hay otros candidatos que se esfuerzan como nosotros en ser los elegidos. ### Fuentes de ofertas -Aunque creo que las grandes plataformas de empleo son menos efectivas que otras vías para encontrar trabajo, sí vale la pena examinar las ofertas para saber qué demanda el mercado y de paso apuntarse a todas las ofertas que podamos. -Esto último igual hasta nos sirve para entrar en algún proceso y quizá conseguir trabajo, pero también es importante porque los procesos de selección requieren práctica real para mejorar nuestras posibilidades. +Aunque creo que las grandes plataformas de empleo son menos efectivas que otras vías para encontrar trabajo, sí vale la pena examinar las ofertas para saber qué demanda el mercado y de paso apuntarse a todas las ofertas que podamos. Esto último igual hasta nos sirve para entrar en algún proceso y quizá conseguir trabajo, pero también es importante porque los procesos de selección requieren práctica real para mejorar nuestras posibilidades. -Es importante recordar que muy frecuentemente lo que parecen requisitos en estas ofertas de empleo no lo son. -Si una empresa pide más conocimientos en una oferta de lo que es razonable, es muy probable que no encuentren a nadie que los cumpla todos y que contraten a alguien que no cumple todos los requisitos. +Es importante recordar que muy frecuentemente lo que parecen requisitos en estas ofertas de empleo no lo son. Si una empresa pide más conocimientos en una oferta de lo que es razonable, es muy probable que no encuentren a nadie que los cumpla todos y que contraten a alguien que no cumple todos los requisitos. En general, el mejor lugar para encontrar mejores vacantes son las pequeñas comunidades: @@ -95,12 +77,9 @@ En general, el mejor lugar para encontrar mejores vacantes son las pequeñas com * Así mismo, muchas tecnologías también tienen sus propias comunidades, aunque haya menos específicamente españolas. (En general, conseguir trabajos en el extranjero es bastante más complicado, así que recomiendo centrarse en comunidades españolas.) -Muchas de estas comunidades tienen tablones de anuncios de ofertas de empleo. -Muchas de estas ofertas las ponen los miembros de la comunidad y no las empresas, con lo que es más probable que sean reales, y en muchos casos, podremos hablar con la persona que pone el anuncio directamente. -Además, en muchos casos los tablones de anuncios de comunidades tienen reglas más estrictas sobre publicación de rangos salariales y claridad en condiciones (como por ejemplo, la modalidad real de remoto). +Muchas de estas comunidades tienen tablones de anuncios de ofertas de empleo. Muchas de estas ofertas las ponen los miembros de la comunidad y no las empresas, con lo que es más probable que sean reales, y en muchos casos, podremos hablar con la persona que pone el anuncio directamente. Además, en muchos casos los tablones de anuncios de comunidades tienen reglas más estrictas sobre publicación de rangos salariales y claridad en condiciones (como por ejemplo, la modalidad real de remoto). -El volumen es por supuesto muy inferior, pero merece mucho la pena encontrar cuantos más tablones de anuncios de este tipo y centrarse más en sus ofertas. -(Aunque raramente tendremos suficiente con estas ofertas para encontrar empleo, con lo que siempre deberemos tirar de las grandes plataformas.) +El volumen es por supuesto muy inferior, pero merece mucho la pena encontrar cuantos más tablones de anuncios de este tipo y centrarse más en sus ofertas. (Aunque raramente tendremos suficiente con estas ofertas para encontrar empleo, con lo que siempre deberemos tirar de las grandes plataformas.) ### El currículum y la presencia online @@ -118,14 +97,13 @@ Un proceso de selección jamás puede evaluar adecuadamente la capacidad de un c Al no ser algo muy exacto, los procesos de selección tienen muchísimo de imitación y modas. -Esto tiene una ventaja; en cualquier momento determinado de tiempo hay como media docena de tipos de entrevista. -Además, para cada tipo de prueba de moda, hay bastantes materiales para preparar la prueba. +Esto tiene una ventaja; en cualquier momento determinado de tiempo hay como media docena de tipos de entrevista. Además, para cada tipo de prueba de moda, hay bastantes materiales para preparar la prueba. En mi opinión, hay que aceptar los sinsentidos de los procesos de selección, dedicar una cantidad significativa de tiempo para prepararnos los pocos formatos más populares en ese momento, y quizá consolarnos con que el número de formatos populares no es mucho mayor. ### Escogiendo empleos -[La falacia de McNamara dice que a la hora de tomar decisiones damos más importancia a lo que es fácil de medir con un número que a lo que no](https://es.wikipedia.org/wiki/Falacia_de_McNamara). +=> https://es.wikipedia.org/wiki/Falacia_de_McNamara La falacia de McNamara dice que a la hora de tomar decisiones damos más importancia a lo que es fácil de medir con un número que a lo que no. El sueldo es de las pocas variables que podemos conocer cuando tenemos una oferta en la mesa, pero no es tan mala métrica. diff --git a/blog/content/notas/tecnologia/problemas.gmi b/blog/content/notas/tecnologia/problemas.gmi new file mode 100644 index 00000000..da4ee14f --- /dev/null +++ b/blog/content/notas/tecnologia/problemas.gmi @@ -0,0 +1,63 @@ +# Problemas + +Este documento es una lista de cosas que me tocan las narices. En el mundo hay infinidad de problemas más graves, pero quiero destacar esta lista. + +## La basura telefónica está fuera de control + +Recibo frecuentemente tanto llamadas como SMS fraudulentos o de publicidad indeseada. Estoy apuntado en la lista Robinson y simplemente no hay manera de librarse. + +No corro un riesgo severo de ser víctima de un fraude, pero imagino que una cantidad importante de gente sí lo corre. (Aunque una vez, la compañía eléctrica de la que era cliente sí me coló un timo.) + +Hasta donde yo sé: + +* Es trivial falsear la identificación de un SMS para que sea idéntico al que envía una entidad legítima. (E.g.: puedes hacer un SMS que se identifique como "Correos", igual que un SMS legítimo de Correos.) +* No existe, hasta donde yo sé, todo mecanismo de denuncia requiere que identifiquemos al autor de la llamada. Identificar quién llama sólo es posible si la empresa decide identificarse. +* No percibo ninguna consecuencia negativa para nadie que participe en la basura telefónica. Esto incluye a quienes realizan estas llamadas y SMS, y a las operadoras telefónicas por las que fluyen. + +Los filtros de Google son moderadamente efectivos, pero no están al alcance de todo el mundo. Además, los falsos positivos de los filtros pueden hacer perdernos comunicaciones legítimas importantes. + +### Recomendaciones frente a la basura telefónica + +A nivel individual, podemos formarnos para hacernos menos vulnerables a los fraudes, pero es prácticamente imposible evitar las molestias. + +Mi recomendación para quien tenga un móvil con posibilidad de reportar llamadas de spam, es coger las llamadas que muestren un número y: + +* Decir algo. Si no decimos nada, muchas centralitas de spam no conectan a un operador, con lo que no podemos tener 100% la certeza de que sea spam. +* Esperar a que respondan para asegurarnos de que es spam. +* Si parece que hay un robot al otro lado de la línea, colgar inmediatamente. Si parece que hay un humano, esperar a que cuelgue. (Con esto, el humano no está libre para hacer otra llamada, con lo que les frenamos un poco. También podemos intentar alargar la llamada, aunque yo personalmente no tengo paciencia.) +* Marcar la llamada como spam. + +Tengo la sensación de que los indicadores de spam de muchos teléfonos se basan en las denuncias que recibe cada número. Así que cuantas más llamadas se cojan y se marquen como spam, antes aparecerán marcadas claramente como spam para otras personas. + +(Es importante que los filtros antispam sean precisos.) + +### Otras referencias + +=> https://www.hiya.com/global-call-threat-report Hiya Global Call Threat Report Q4 2024 + +## Los protocolos cerrados dan un poder desproporcionado a empresas privadas + +Para la mayoría de gente, WhatsApp es prácticamente una necesidad para la vida cotidiana. + +Esto hace que Meta controle una parte sustancial de nuestras comunicaciones, queramos o no. Además, esto hace que cualquier problema con WhatsApp (incidencia, carencia, etc.) sea inevitable. + +También hay efectos inesperados como que Meta decide qué sistemas operativos móviles son viables y cuáles no. (En un par de ocasiones, me he tenido que cambiar de móvil porque Meta ha decidido dejar de soportarlo. Aunque puedo experimentar con sistemas operativos móviles alternativos, siempre tengo que tener un móvil soportado por WhatsApp.) + +La ley de mercados digitales (DMA) en teoría ayudará parcialmente. Esta ley debería obligar a WhatsApp a interoperar, con lo que podríamos comunicarnos con usuarios de WhatsApp sin usar WhatsApp, mitigando algunos problemas. Sin embargo, aunque lleva en vigor desde el 7 de marzo de 2024, esto todavía no es posible y está por ver cuán efectivo será. + +Muchos sistemas de comunicación existentes son más abiertos que los protocolos modernos: + +* Cualquiera puede montar un servidor de correo y comunicarse con usuarios de correo electrónico de otros proveedores. Pese a que muchos apuntan a que Google y Microsoft tienen un poder desproporcionado de facto, sigue siendo totalmente viable usar otros proveedores. Y aunque se apunta que la interoperabilidad de los correos es causante del spam, muchos otros sistemas cerrados como WhatsApp tienen problemas de spam similares o mayores. +* Aunque no todo el mundo puede hacer emisiones de DVB-T, cualquiera con una antena puede captar las emisiones y visualizarlas. (Los protocolos con los que se codifican las emisiones de DVB-T están disponibles para cualquiera.) También es posible codificar las emisiones de DVB-T para limitar su uso a usuarios que paguen, pero con libertad de consumir los contenidos con cualquier sistema DVB-T de nuestro agrado. + +(Esto en contraste con los servicios de streaming, que sólo podemos usar con dispositivos validados por el servicio de streaming.) + +## Los navegadores son excesivamente complejos + +Gran parte de los contenidos y procesos que tenemos que realizar hoy en día pasan por un navegador web. + +Por diversos motivos, los navegadores cada vez son más sofisticados para permitir mayores funcionalidades. Son tan complejos que Microsoft, una de las mayores empresas tecnológicas del mundo, ha renunciado a desarrollar un navegador propio y reutiliza gran parte de Chrome, un navegador controlado por una empresa con la que compite, Google. + +Fuera de Google Chrome y de Safari de Apple, virtualmente no existen navegadores que compitan con ellos. (Hay más navegadores, pero como Edge de Microsoft, usan el motor de Chrome o de Safari. Firefox es cada vez más minoritario e irrelevante [aunque yo lo uso y animo a todo el mundo a que lo use].) + +La sofisticación y complejidad de Chrome y Safari adicionalmente hacen que cada vez existan más webs y aplicaciones web que son prácticamente inutilizables en dispositivos de rendimiento modesto. Esto hace que sea virtualmente necesario renovar nuestros dispositivos con más frecuencia de la necesaria, a dispositivos más costosos de lo que necesitaríamos para el resto de nuestros propósitos. diff --git a/linux/quiero_instalar_linux.md b/blog/content/notas/tecnologia/quiero-instalar-linux.gmi index 8caa7e17..e8a5e183 100644 --- a/linux/quiero_instalar_linux.md +++ b/blog/content/notas/tecnologia/quiero-instalar-linux.gmi @@ -8,90 +8,72 @@ En este documento recojo mis consejos para los que estáis en esta situación. ## No es necesario pasarse a Linux -Vaya por delante que aunque llevo usando Linux en mis ordenadores personales desde los inicios de siglo, y laboralmente más de una década. -Linux es indudablemente uno de los mejores sistemas operativos que podemos usar en nuestros ordenadores personales, especialmente si creemos que es mejor para nuestros intereses usar software libre. +Vaya por delante que aunque llevo usando Linux en mis ordenadores personales desde los inicios de siglo, y laboralmente más de una década. Linux es indudablemente uno de los mejores sistemas operativos que podemos usar en nuestros ordenadores personales, especialmente si creemos que es mejor para nuestros intereses usar software libre. Sin embargo: * Usar Linux sigue siendo algo más complicado que usar Windows, macOS, ChromeOS, Android y otros. -* Todos los fabricantes de ordenadores y componentes dedican un esfuerzo razonable a hacer funcionar sus productos en Windows. - Muy pocos fabricantes hacen lo mismo para Linux. - Aunque muchas veces los desarrolladores de Linux consiguen hacer funcionar hardware, escoger hardware con buen soporte para Linux es prácticamente indispensable, y esto no garantiza una experiencia libre de problemas. +* Todos los fabricantes de ordenadores y componentes dedican un esfuerzo razonable a hacer funcionar sus productos en Windows. Muy pocos fabricantes hacen lo mismo para Linux. Aunque muchas veces los desarrolladores de Linux consiguen hacer funcionar hardware, escoger hardware con buen soporte para Linux es prácticamente indispensable, y esto no garantiza una experiencia libre de problemas. -Aunque uso Linux, he usado Windows, macOS, ChromeOS y Android, y es perfectamente lícito usarlos. -Usando Linux seguramente aprendáis cosas nuevas y obtengáis beneficios, pero no siempre es lo mejor que podéis hacer con vuestro tiempo libre. +Aunque uso Linux, he usado Windows, macOS, ChromeOS y Android, y es perfectamente lícito usarlos. Usando Linux seguramente aprendáis cosas nuevas y obtengáis beneficios, pero no siempre es lo mejor que podéis hacer con vuestro tiempo libre. Sed conscientes que pasaros a Linux requerirá esfuerzo, tiempo e incluso dinero. ## No es necesario pasar nuestro único ordenador a Linux -La mejor manera de instalar Linux en la mayoría de casos es reemplazar el sistema operativo de un ordenador completamente. -(Se puede instalar Linux en paralelo a Windows, por ejemplo, y esto tiene ventajas, pero también es una fuente de quebraderos de cabeza.) +La mejor manera de instalar Linux en la mayoría de casos es reemplazar el sistema operativo de un ordenador completamente. (Se puede instalar Linux en paralelo a Windows, por ejemplo, y esto tiene ventajas, pero también es una fuente de quebraderos de cabeza.) -Esto quiere decir que hasta que no tengamos Linux funcionando, el ordenador donde estemos jugando seguramente estará inservible. -Podemos tener problemas de instalación que nos impidan usar Linux, o incluso una vez instalado, podría no gustarnos. +Esto quiere decir que hasta que no tengamos Linux funcionando, el ordenador donde estemos jugando seguramente estará inservible. Podemos tener problemas de instalación que nos impidan usar Linux, o incluso una vez instalado, podría no gustarnos. Para hacer pruebas, recomiendo siempre considerar las siguientes posibilidades: -* Probar Linux en una máquina virtual. +### Probar Linux en una máquina virtual - La mayoría de ordenadores hoy en día tienen suficientes recursos para crear una máquina virtual. - Una máquina virtual es un ordenador dentro de nuestro ordenador donde podemos instalar un sistema operativo distinto. +La mayoría de ordenadores hoy en día tienen suficientes recursos para crear una máquina virtual. Una máquina virtual es un ordenador dentro de nuestro ordenador donde podemos instalar un sistema operativo distinto. - Instalar Linux en una máquina virtual es más sencillo que instalarlo en hardware real (en general, nunca hay problemas de compatibilidad de hardware). - Aunque el rendimiento de Linux será menor al que tendrá instalado directamente en el hardware, nos permite hacernos una idea de lo que implica usar Linux antes de dar el salto. +Instalar Linux en una máquina virtual es más sencillo que instalarlo en hardware real (en general, nunca hay problemas de compatibilidad de hardware). Aunque el rendimiento de Linux será menor al que tendrá instalado directamente en el hardware, nos permite hacernos una idea de lo que implica usar Linux antes de dar el salto. -* Usar un ordenador secundario del que no dependemos. +### Usar un ordenador secundario del que no dependemos - En general, es posible encontrar ordenadores que no están en uso. - Recomiendo cualquier ordenador con cuatro gigas de RAM y un SSD para hacer pruebas. +En general, es posible encontrar ordenadores que no están en uso. Recomiendo cualquier ordenador con cuatro gigas de RAM y un SSD para hacer pruebas. - Además, tener un ordenador para hacer pruebas siempre nos servirá para otra cosa, como sustituir temporalmente nuestro ordenador principal averiado u otros. +Además, tener un ordenador para hacer pruebas siempre nos servirá para otra cosa, como sustituir temporalmente nuestro ordenador principal averiado u otros. - Si no disponemos de uno, considerar examinar las tiendas de segunda mano. - No suele ser complicado encontrar ordenadores de segunda mano usables a un precio asequible, incluso de modelos cuyo fabricante soporta el uso de Linux, como algunos ThinkPads, HP y Dell. +Si no disponemos de uno, considerar examinar las tiendas de segunda mano. No suele ser complicado encontrar ordenadores de segunda mano usables a un precio asequible, incluso de modelos cuyo fabricante soporta el uso de Linux, como algunos ThinkPads, HP y Dell. -* Probar Linux "live". +### Probar Linux "live" - La mayoría de distribuciones Linux tienen una versión "live" que permite ejecutar Linux desde un disco USB sin modificar nuestra instalación existente. - Esto es más conveniente para probar si un ordenador del que disponemos tiene buen soporte de Linux, pero también nos sirve para hacernos una idea del funcionamiento de Linux. +La mayoría de distribuciones Linux tienen una versión "live" que permite ejecutar Linux desde un disco USB sin modificar nuestra instalación existente. Esto es más conveniente para probar si un ordenador del que disponemos tiene buen soporte de Linux, pero también nos sirve para hacernos una idea del funcionamiento de Linux. -* Usar discos duros totalmente separados para diferentes sistemas operativos. +### Usar discos duros totalmente separados para diferentes sistemas operativos - La mayoría de sistemas permiten añadir un segundo disco duro y escoger cuál usar al arrancar el ordenador. - Esto tiene un coste extra, y en muchos casos como la mayoría de los portátiles, añade la inconveniencia de un disco duro externo. - Sin embargo, elimina la mayoría de los problemas del arranque dual e incluso los dispositivos separados (y especialmente los externos) nos pueden dar un plus de flexibilidad. +La mayoría de sistemas permiten añadir un segundo disco duro y escoger cuál usar al arrancar el ordenador. Esto tiene un coste extra, y en muchos casos como la mayoría de los portátiles, añade la inconveniencia de un disco duro externo. Sin embargo, elimina la mayoría de los problemas del arranque dual e incluso los dispositivos separados (y especialmente los externos) nos pueden dar un plus de flexibilidad. - (Es recomendable siempre que los sistemas operativos estén en un disco SSD conectado mediante un interfaz rápido, evitando versiones de USB anteriores a USB 3.) +(Es recomendable siempre que los sistemas operativos estén en un disco SSD conectado mediante un interfaz rápido, evitando versiones de USB anteriores a USB 3.) Para un uso duradero de Linux, en general tampoco recomiendo el arranque dual, por ser una potencial fuente de problemas de fiabilidad difíciles de resolver. -El arranque dual permite que los dos sistemas operativos funcionen directamente sobre el hardware, lo que puede ser necesario para que el software en *ambos* sistemas operativos pueda acceder a todo el hardware al máximo rendimiento. -Sin embargo, la mayoría de sistemas operativos no están pensados para el arranque dual y, por ejemplo las actualizaciones pueden estropear el arranque dual. +El arranque dual permite que los dos sistemas operativos funcionen directamente sobre el hardware, lo que puede ser necesario para que el software en *ambos* sistemas operativos pueda acceder a todo el hardware al máximo rendimiento. Sin embargo, la mayoría de sistemas operativos no están pensados para el arranque dual y, por ejemplo las actualizaciones pueden estropear el arranque dual. -Para un uso duradero de Linux, a parte de usar hardware separado, podemos considerar: +### Usar máquinas virtuales -* El uso de máquinas virtuales, que adicionalmente pueden evitar problemas de compatibilidad de hardware. - (No todo el hardware funciona con Linux, pero Linux casi siempre funciona perfectamente en una máquina virtual.) +Para un uso duradero de Linux, a parte de usar hardware separado, podemos considerar el uso de máquinas virtuales, que adicionalmente pueden evitar problemas de compatibilidad de hardware. (No todo el hardware funciona con Linux, pero Linux casi siempre funciona perfectamente en una máquina virtual.) - Windows además dispone de WSL, que es un mecanismo para integrar una máquina virtual Linux con Windows. +Windows además dispone de WSL, que es un mecanismo para integrar una máquina virtual Linux con Windows. - También podemos correr Windows dentro de una máquina virtual en Linux, aunque la licencia que teníamos de Windows en general no es apta para este uso. +También podemos correr Windows dentro de una máquina virtual en Linux, aunque la licencia que teníamos de Windows en general no es apta para este uso. - Las máquinas virtuales tienen menor rendimiento y desventajas respecto a correr en el hardware directamente, aunque tienen otras ventajas. - (Por ejemplo, es muy fácil hacer una copia de respaldo de una máquina virtual, en general podemos mover máquinas virtuales de un sistema a otro, y más.) +Las máquinas virtuales tienen menor rendimiento y desventajas respecto a correr en el hardware directamente, aunque tienen otras ventajas. (Por ejemplo, es muy fácil hacer una copia de respaldo de una máquina virtual, en general podemos mover máquinas virtuales de un sistema a otro, y más.) -* La mayoría de software disponible en Linux se puede usar en otros sistemas operativos. - Incluso hay proyectos específicos para ofrecer plataformas similares a Linux dentro de otros sistemas operativos. +### Usar software libre on otros sistemas operativos + +La mayoría de software disponible en Linux se puede usar en otros sistemas operativos. Incluso hay proyectos específicos para ofrecer plataformas similares a Linux dentro de otros sistemas operativos. ## Como estar preparados para trastear con Linux -Siempre que uno utiliza un ordenador es importante tener bien controlados nuestros datos. -Esto es todavía más importante si queremos experimentar con distintos sistemas operativos como Linux. +Siempre que uno utiliza un ordenador es importante tener bien controlados nuestros datos. Esto es todavía más importante si queremos experimentar con distintos sistemas operativos como Linux. -Es recomendable que no tengamos ningún dato que solamente resida en un ordenador. -En condiciones normales, esto implica que un fallo de disco nos puede hacer perder datos valiosos. -Instalando Linux, en general, borraremos el disco o haremos operaciones que pueden hacer que perdamos estos datos temporal o permanentemente. +Es recomendable que no tengamos ningún dato que solamente resida en un ordenador. En condiciones normales, esto implica que un fallo de disco nos puede hacer perder datos valiosos. Instalando Linux, en general, borraremos el disco o haremos operaciones que pueden hacer que perdamos estos datos temporal o permanentemente. Es recomendable siempre tener varias copias de nuestros datos importantes, sea en la nube o en medios que tengamos físicamente. @@ -101,9 +83,7 @@ Siempre es recomendable tener claro que podemos reinstalar o perder un ordenador Y en caso de necesidad, siempre es bueno conocer el proceso de reinstalación de Windows. -Muchos fabricantes incluyen una funcionalidad de recuperación de Windows que podemos conservar al instalar Linux (pero hay que prestar atención). -También suelen ofrecer descargar el instalador de Windows. -(Nunca está de más conocer el proceso de recuperación o instalación de Windows, pues siempre lo podemos necesitar sin ser usuarios de Linux.) +Muchos fabricantes incluyen una funcionalidad de recuperación de Windows que podemos conservar al instalar Linux (pero hay que prestar atención). También suelen ofrecer descargar el instalador de Windows. (Nunca está de más conocer el proceso de recuperación o instalación de Windows, pues siempre lo podemos necesitar sin ser usuarios de Linux.) ## No todos los ordenadores soportan bien Linux @@ -115,15 +95,14 @@ Obviamente, con la inmensa variedad de ordenadores en el mercado, esto no siempr Algunas empresas que trabajan con Linux prueban modelos de ordenador en colaboración algunos fabricantes y publican listas de ordenadores certificados, como por ejemplo: -* [Ubuntu](https://ubuntu.com/certified) -* [Red Hat](https://catalog.redhat.com/search?searchType=hardware&type=System&system_types=Desktop%2FWorkstation|Laptop&p=1) +=> https://ubuntu.com/certified Ubuntu +=> https://catalog.redhat.com/search?searchType=hardware&type=System&system_types=Desktop%2FWorkstation|Laptop&p=1 Red Hat Así mismo, algunos fabricantes venden modelos con Linux preinstalados. Por supuesto, puede ser ventajoso poder reciclar hardware que ya tenemos, pero hoy en día hay bastante hardware con soporte para Linux y suele ser siempre una buena opción. -Como mencionábamos anteriormente, siempre antes de instalar es recomendable probar una versión "live" de Linux para comprobar si el hardware funciona bien en Linux. -Es recomendable siempre comprobar: +Como mencionábamos anteriormente, siempre antes de instalar es recomendable probar una versión "live" de Linux para comprobar si el hardware funciona bien en Linux. Es recomendable siempre comprobar: * Conexión a la red: deberíamos poder acceder a Internet con o sin cable según sea nuestra preferencia * Sonido: que funcionen bien las salidas de audio que usemos (tomas de auriculares, altavoces, etc.). @@ -147,17 +126,16 @@ Sin embargo, hay que considerar que hay diversos grados de dificultad en las dis Hay una serie de criterios a observar al escoger una distribución: * Que tenga versión "live" para poder probar nuestro hardware. -* Que sea muy popular en Internet. - Muchos problemas tienen soluciones distintas según la distribución que usemos. - Siempre es más fácil seguir instrucciones para la misma distribución. -* Diferentes distribuciones tienen políticas distintas sobre distribuir software no libre: +* Que sea muy popular en Internet. Muchos problemas tienen soluciones distintas según la distribución que usemos. Siempre es más fácil seguir instrucciones para la misma distribución. + +Diferentes distribuciones tienen políticas distintas sobre distribuir software no libre: - * La distribución distribuye software no libre sin diferenciarlo del software libre. - * La distribución distribuye software no libre, aunque haya que activarlo o hacer alguna pequeña configuración. - * La distribución no distribuye software no libre, pero hay entidades que empaquetan software no libre para esta distribución. - * La distribución no distribuye software no libre, no hay entidades que empaqueten software no libre para esta distribución, o hay cierta hostilidad al uso de software no libre entre la comunidad. +* La distribución distribuye software no libre sin diferenciarlo del software libre. +* La distribución distribuye software no libre, aunque haya que activarlo o hacer alguna pequeña configuración. +* La distribución no distribuye software no libre, pero hay entidades que empaquetan software no libre para esta distribución. +* La distribución no distribuye software no libre, no hay entidades que empaqueten software no libre para esta distribución, o hay cierta hostilidad al uso de software no libre entre la comunidad. - Esto puede no parecer importante, pero es más común de lo que parece que ciertos dispositivos hardware o algunas funcionalidades dependan de software no libre y esto nos suponga un problema en ciertas distribuciones. +Esto puede no parecer importante, pero es más común de lo que parece que ciertos dispositivos hardware o algunas funcionalidades dependan de software no libre y esto nos suponga un problema en ciertas distribuciones. Otro criterio importante pero más complejo es la política de versiones de la distribución. @@ -165,8 +143,7 @@ Otro criterio importante pero más complejo es la política de versiones de la d Hay distribuciones de soporte prolongado, como Ubuntu (LTS), Debian (stable) o Red Hat Enterprise Linux (y derivados) que sacan una versión nueva cada 2-3 años y se comprometen a soportar esa versión durante un periodo largo de tiempo. -Esto quiere decir que el software que llevan puede no estar muy al día y que podemos tener problemas de soporte de hardware, especialmente para hardware muy moderno. -Sin embargo, respecto a distribuciones con ciclos de vida más cortos, ofrece la gran ventaja de que las actualizaciones rutinarias son más sencillas y conllevan menos riesgo de problemas. +Esto quiere decir que el software que llevan puede no estar muy al día y que podemos tener problemas de soporte de hardware, especialmente para hardware muy moderno. Sin embargo, respecto a distribuciones con ciclos de vida más cortos, ofrece la gran ventaja de que las actualizaciones rutinarias son más sencillas y conllevan menos riesgo de problemas. Si encontramos una distribución Linux con soporte prolongado que soporta bien nuestro hardware, recomiendo comenzar con una distribución así mientras aprendemos Linux. @@ -194,28 +171,20 @@ Por lo general, hoy en día todas las distribuciones Linux contienen instruccion Estas instrucciones, si no queremos complicarnos la vida, borran el contenido del USB y lo reemplazan completamente, con lo que debemos usar un USB que no contenga datos importantes. -La mayoría de instaladores tienen funcionalidad "live". -Es muy recomendable probar todas las funcionalidades que podamos con la versión "live" de la manera más exhaustiva posible antes de hacer la instalación definitiva. +La mayoría de instaladores tienen funcionalidad "live". Es muy recomendable probar todas las funcionalidades que podamos con la versión "live" de la manera más exhaustiva posible antes de hacer la instalación definitiva. Si el instalador no es "live", normalmente podremos encontrar una versión alternativa para la misma distribución, o en el peor caso, probar con el "live" de otra distribución. La distribución debería documentar adecuadamente el proceso de instalación, pero casi siempre consiste en: -* Configurar el ordenador para arrancar el instalador en vez del sistema operativo que ya tenemos instalado. - Esto en general se hace con lo que se suele llamar "BIOS", a la que accedemos pulsando una tecla específica durante el proceso de arranque. - Si nuestro sistema actual es Windows, las versiones modernas tienen funcionalidades de arranque rápido que pueden interferir con la selección de arranque. - Es posible que necesitemos consultar instrucciones más específicas para cambiar el arranque. - +* Configurar el ordenador para arrancar el instalador en vez del sistema operativo que ya tenemos instalado. Esto en general se hace con lo que se suele llamar "BIOS", a la que accedemos pulsando una tecla específica durante el proceso de arranque. Si nuestro sistema actual es Windows, las versiones modernas tienen funcionalidades de arranque rápido que pueden interferir con la selección de arranque. Es posible que necesitemos consultar instrucciones más específicas para cambiar el arranque. * Probar el sistema en el entorno "live" y luego seleccionar una opción para hacer la instalación definitiva. - * Escoger el nombre del ordenador, de nuestro usuario y otras opciones durante el proceso de instalación. +* Una opción importante es si queremos que Linux y nuestro sistema operativo existente (típicamente Windows) coexistan o reemplazar el sistema operativo existente completamente con Linux. Anteriormente en este documento se detallan recomendaciones sobre evitar la coexistencia de varios sistemas operativos en un mismo disco duro. -* Una opción importante es si queremos que Linux y nuestro sistema operativo existente (típicamente Windows) coexistan o reemplazar el sistema operativo existente completamente con Linux. - Anteriormente en este documento se detallan recomendaciones sobre evitar la coexistencia de varios sistemas operativos en un mismo disco duro. - -### ¿Qué problemas puede haber? +## ¿Qué problemas puede haber? -#### Problemas de medios USB +### Problemas de medios USB Los discos USB tienen un amplio abanico de calidades. @@ -228,74 +197,57 @@ Ante problemas extraños, siempre es recomendable: * Verificar la descarga siguiendo las instrucciones de la distribución. * Verificar el disco USB; muchos instaladores permiten hacer una verificación antes de empezar, algunos programas para grabar USB pueden hacer una comprobación después de grabar. -Aunque las verificaciones no encuentren problemas, sigue habiendo margen para el error. -Siempre es conveniente tener otro USB para hacer otra prueba, para descartar problemas. +Aunque las verificaciones no encuentren problemas, sigue habiendo margen para el error. Siempre es conveniente tener otro USB para hacer otra prueba, para descartar problemas. -#### Problemas de instalación +### Problemas de instalación -Incluso con un medio correcto, la instalación puede fallar. -Los instaladores pueden tener defectos inesperados o podemos toparnos con problemas temporales de la infraestructura de la distribución que impidan la instalación. +Incluso con un medio correcto, la instalación puede fallar. Los instaladores pueden tener defectos inesperados o podemos toparnos con problemas temporales de la infraestructura de la distribución que impidan la instalación. Una vez descartamos problemas de medios, es recomendable probar con otra distribución. Tampoco es imposible que simplemente el ordenador que estemos usando tenga algún problema de hardware preexistente que sólo se manifieste durante el proceso de instalación, por someter al ordenador a una carga no habitual. -En general, todos los instaladores de Linux dan actualizaciones constantes del avance de la instalación. -Puede que el sistema se bloquee o no responda fluidamente, pero nunca más de unos minutos. +En general, todos los instaladores de Linux dan actualizaciones constantes del avance de la instalación. Puede que el sistema se bloquee o no responda fluidamente, pero nunca más de unos minutos. -Si el sistema no responde durante varios minutos, tendremos que reiniciar el ordenador a la fuerza. -Si los problemas se repiten, lo mejor es probar con otro disco USB, verificar la descarga del instalador, o probar con otra distribución. +Si el sistema no responde durante varios minutos, tendremos que reiniciar el ordenador a la fuerza. Si los problemas se repiten, lo mejor es probar con otro disco USB, verificar la descarga del instalador, o probar con otra distribución. -(Tendremos que volver a seleccionar el arranque del instalador. -Este proceso puede ser distinto; si antes lo hicimos desde Windows, una vez reemplazado Windows, tendremos que usar "la BIOS".) +(Tendremos que volver a seleccionar el arranque del instalador. Este proceso puede ser distinto; si antes lo hicimos desde Windows, una vez reemplazado Windows, tendremos que usar "la BIOS".) -### ¿Y si no me gusta? +## ¿Y si no me gusta? -Como mencionamos anteriormente, siempre debería ser posible volver a Windows. -Si nos hemos asegurado que nuestros datos estén protegidos, podemos reinstalar Windows de nuevo o usar las funcionalidades de recuperación. +Como mencionamos anteriormente, siempre debería ser posible volver a Windows. Si nos hemos asegurado que nuestros datos estén protegidos, podemos reinstalar Windows de nuevo o usar las funcionalidades de recuperación. ## Instalando programas en Linux -Un ordenador y un sistema operativo no son más que medios para usar programas. -De hecho, las distribuciones de Linux se llaman distribuciones porque aparte de Linux en sí (que por sí mismo no sirve de mucho), empaquetan una colección de programas para que nuestro ordenador sea útil. +Un ordenador y un sistema operativo no son más que medios para usar programas. De hecho, las distribuciones de Linux se llaman distribuciones porque aparte de Linux en sí (que por sí mismo no sirve de mucho), empaquetan una colección de programas para que nuestro ordenador sea útil. Por tanto, es importante saber qué programas necesitamos y cómo instalarlos en Linux. -Lo más fácil suele ser instalar los programas que empaqueta nuestra distribución. -Además, las actualizaciones de los programas instalados desde nuestra distribución están integradas con las actualizaciones del sistema, lo que supone una ventaja interesante respecto a otros sistemas operativos. - -Sin embargo, ni la distribución más extensa puede contener todos los programas que pueda necesitar todo el mundo. -(Aunque en general, cualquiera puede pasar por un proceso para añadir el programa que quiera a una distribución, aunque en general empaquetar programas para distribuciones es de las maneras más complejas de distribuir software.) +Lo más fácil suele ser instalar los programas que empaqueta nuestra distribución. Además, las actualizaciones de los programas instalados desde nuestra distribución están integradas con las actualizaciones del sistema, lo que supone una ventaja interesante respecto a otros sistemas operativos. -Adicionalmente, las distribuciones (especialmente las de soporte prolongado[^1]) no siempre están al día con las últimas versiones del software que empaquetan. +Sin embargo, ni la distribución más extensa puede contener todos los programas que pueda necesitar todo el mundo. (Aunque en general, cualquiera puede pasar por un proceso para añadir el programa que quiera a una distribución, aunque en general empaquetar programas para distribuciones es de las maneras más complejas de distribuir software.) -[^1]: las distribuciones de soporte prolongado y muchas de soporte breve sólo incorporan actualizaciones de seguridad y corrección de defectos. - Aunque esto supone sus ventajas, muchas veces nos impedirá tener acceso a las funcionalidades más nuevas de un programa. +Adicionalmente, las distribuciones (especialmente las de soporte prolongado) no siempre están al día con las últimas versiones del software que empaquetan. (Las distribuciones de soporte prolongado y muchas de soporte breve sólo incorporan actualizaciones de seguridad y corrección de defectos. Aunque esto supone sus ventajas, muchas veces nos impedirá tener acceso a las funcionalidades más nuevas de un programa.) La mayoría de distribuciones tienen páginas web donde podemos buscar un programa y ver qué versiones están empaquetadas en qué versión de la distribución, como por ejemplo: -* [Debian](https://www.debian.org/distrib/packages) -* [Ubuntu](https://packages.ubuntu.com/) -* [Fedora](https://packages.fedoraproject.org/) +=> https://www.debian.org/distrib/packages Debian +=> https://packages.ubuntu.com/ Ubuntu +=> https://packages.fedoraproject.org/ Fedora -Nota: los paquetes de una distribución en general sólo se deben usar en la misma distribución *y* versión de la distribución para la que está hecho el paquete. -Aunque a veces es posible aprovechar paquetes, no es recomendable si no sabemos bien cómo funciona. +Nota: los paquetes de una distribución en general sólo se deben usar en la misma distribución *y* versión de la distribución para la que está hecho el paquete. Aunque a veces es posible aprovechar paquetes, no es recomendable si no sabemos bien cómo funciona. Pero existen más mecanismos para instalar software. ### Sistemas de paquetes genéricos -Principalmente Flatpak y Snap son dos sistemas para instalar software que funcionan en muchas distribuciones. -A diferencia de los paquetes de una distribución, que sólo funcionan en esa distribución, un Flatpak o un Snap funcionarán en todas las distribuciones que estén soportadas ([aquí las distribuciones que soporta Flatpak](https://flatpak.org/setup/ "Instrucciones para instalar Flatpak en distribuciones soportadas") y [las soportadas por Snap](https://snapcraft.io/docs/installing-snapd "Instrucciones para instalar Snap en distribuciones soportadas")). +Principalmente Flatpak y Snap son dos sistemas para instalar software que funcionan en muchas distribuciones. A diferencia de los paquetes de una distribución, que sólo funcionan en esa distribución, un Flatpak o un Snap funcionarán en todas las distribuciones que estén soportadas. -La ventaja indirecta para el usuario es que el autor de un programa (o cualquiera) puede hacer un Flatpak o un Snap e inmediatamente lo pueden usar usuarios de distintas distribuciones, sin tener que hacer el esfuerzo de empaquetarlo para todas las distribuciones. -Con esto, hay bastantes programas que están mejor mantenidos como Flatpak o Snap que en las distribuciones. +La ventaja indirecta para el usuario es que el autor de un programa (o cualquiera) puede hacer un Flatpak o un Snap e inmediatamente lo pueden usar usuarios de distintas distribuciones, sin tener que hacer el esfuerzo de empaquetarlo para todas las distribuciones. Con esto, hay bastantes programas que están mejor mantenidos como Flatpak o Snap que en las distribuciones. Sin embargo, mientras que ya confiamos implícitamente en nuestra distribución para tener programas libres de malicia (o al menos, si distribuyen programas maliciosos ya lo sufrimos), al añadir Flatpaks o Snaps debemos ser más cuidadosos, porque los programas pueden quedar desactualizados, el empaquetador no necesariamente es el autor del software, y un largo etcétera. -Flatpak y Snap incorporan mecanismos para aislar los programas de manera que no puedan tener efectos negativos sobre nuestro sistema (como código malicioso, por ejemplo), que pueden ser más avanzados que los mecanismos de protección de la mayoría de distribuciones. -Esto nos da cierta tranquilidad, pero puede causar problemas. -Tanto el hecho de que los Flatpak y Snaps funcionan bajo varias distribuciones como estos mecanismos de protección pueden causar algunos problemas en algunos programas. +Flatpak y Snap incorporan mecanismos para aislar los programas de manera que no puedan tener efectos negativos sobre nuestro sistema (como código malicioso, por ejemplo), que pueden ser más avanzados que los mecanismos de protección de la mayoría de distribuciones. Esto nos da cierta tranquilidad, pero puede causar problemas. Tanto el hecho de que los Flatpak y Snaps funcionan bajo varias distribuciones como estos mecanismos de protección pueden causar algunos problemas en algunos programas. A menudo pero no siempre, los Flatpak y Snap están integrados con las actualizaciones del sistema. @@ -307,8 +259,7 @@ Algunos programas proporcionan repositorios para algunas distribuciones, y hay g Usando estos repositorios, casi siempre conservamos que las actualizaciones de programas de estos repositorios se integran en las actualizaciones del sistema, con lo que es sencillo estar actualizado. -Cuando el repositorio está empaquetado por los autores del programa, en general mejora la confiabilidad del paquete. -En repositorios alternativos, nos tenemos que fiar más de quién mantiene el repositorio, aunque hay repositorios de terceros con un largo historial de confiabilidad y calidad. +Cuando el repositorio está empaquetado por los autores del programa, en general mejora la confiabilidad del paquete. En repositorios alternativos, nos tenemos que fiar más de quién mantiene el repositorio, aunque hay repositorios de terceros con un largo historial de confiabilidad y calidad. ### Distribuciones binarias diff --git a/misc/greek-task-list.md b/blog/content/notes/greek-task-list.gmi index 6979082e..841a6a84 100644 --- a/misc/greek-task-list.md +++ b/blog/content/notes/greek-task-list.gmi @@ -1,83 +1,83 @@ # Greek task list -## From <https://apollos-boyfriend.tumblr.com/post/734812687645786112> +=> https://apollos-boyfriend.tumblr.com/post/734812687645786112 From: -### Icarian task +## Icarian task When you have a task you know you’re going to fail at anyways, so why not have some fun with it before it all comes crashing down -### Cassandrean task +## Cassandrean task When you have to deal with people you KNOW won’t listen to you, despite having accurate information, and having to watch them fumble about when you told them the solution from the start (most often witnessed in customer service) Patroclean task: something that you have a wild misunderstanding of the difficulty involved in accomplishing properly and can only end terribly, but you dive into with your whole heart and the deep faith and conviction that you can accomplish it by sheer force of will. -## From <https://www.tumblr.com/bigdickfartsapolka/734815719754170368/achillean-task-something-you-are-absolutely-and> +=> https://www.tumblr.com/bigdickfartsapolka/734815719754170368/achillean-task-something-you-are-absolutely-and From: -### Patroclean task +## Patroclean task Something that you have a wild misunderstanding of the difficulty involved in accomplishing properly and can only end terribly, but you dive into with your whole heart and the deep faith and conviction that you can accomplish it by sheer force of will. -## From <https://www.tumblr.com/sapphoandvanzetti/791697593363218432/penelopean-task-you-dont-actually-want-the> +=> https://www.tumblr.com/sapphoandvanzetti/791697593363218432/penelopean-task-you-dont-actually-want-the From: -### Penelopean task +## Penelopean task You don't actually want the outcome of the task, so you're self-sabotaging in a clever and undetectable way to keep it from ever being completed. -## From <https://infosec.exchange/users/masek/statuses/115507451151625748> +=> https://infosec.exchange/users/masek/statuses/115507451151625748 From: -### Orphean task +## Orphean task When you almost succeed, but lose everything the moment you turn around to check your progress. -### Daedalean task +## Daedalean task When you’re forced to design something brilliant and functional… that you yourself will inevitably become trapped inside. -### Medusan task +## Medusan task When your project becomes so horrifying that everyone involved freezes in place rather than deal with it. -### Tantaline task +## Tantaline task When success is right there, but bureaucracy or budget cuts keep snatching it away at the last moment, forever. -### Pandoran task +## Pandoran task When fixing one small issue unleashes a thousand new ones, but hey — at least there’s still hope somewhere in the ticket backlog. -### Odyssean task +## Odyssean task When the assignment technically has an end, but it’s buried under so many side quests that you forget what the original goal was. -### Narcissian task +## Narcissian task When the entire effort is about maintaining appearances rather than achieving anything of substance. -### Promethean task +## Promethean task When you give people a powerful new tool that could transform their work — and are punished eternally for doing so. -### Orestian task +## Orestian task When the mess you’re cleaning up is the direct result of the last cleanup you performed. -### Thesean task +## Thesean task When the only way to finish is to disassemble everything piece by piece — until you’re no longer sure if what’s left is the same project you started. -### Achillean task +## Achillean task When your work is flawless except for that one fatal oversight that will, inevitably, destroy you. -### Penelopean task +## Penelopean task When you diligently undo by night what you accomplish by day, just to keep the stakeholders pacified. -### Midasean task +## Midasean task When everything you touch turns into paperwork, compliance documents, or gold-plated nonsense nobody actually needs. -### Gordian task +## Gordian task Not intended to be actually done, but violence is the answer. diff --git a/blog/content/notes/index.gmi b/blog/content/notes/index.gmi index fc971e87..15dc5db1 100644 --- a/blog/content/notes/index.gmi +++ b/blog/content/notes/index.gmi @@ -1,5 +1,7 @@ # Notes +=> greek-task-list Greek task list + ## Cliff's notes Notes about some books I like: @@ -10,6 +12,28 @@ Notes about some books I like: ## Tech +=> tech/misc-linux-stuff Misc Linux stuff +=> tech/running-commands-in-linux Running commands in Linux +=> tech/ssh-for-beginners SSH for beginners +=> tech/document-formats Document formats +=> tech/internet-communication-channels Internet communication channels +=> tech/ledger Ledger +=> tech/migadu Migadu +=> tech/ripping Ripping +=> tech/about-apis About APIs +=> tech/about-relational-databases About relational databases +=> tech/containers-might-not-be-the-right-answer Containers might not be the right answer +=> tech/crud-is-an-important-unsolved-problem CRUD is an important unsolved problem +=> tech/about-django About Django +=> tech/git-advice Git advice +=> tech/github-annoyances GitHub annoyances +=> tech/prolog-vs-sql Prolog vs. SQL +=> tech/take-the-less-traveled-road Take the less traveled road +=> tech/the-tragedy-of-the-geeks The tragedy of the geeks +=> tech/misc-python-stuff Misc Python stuff +=> tech/python-modules-primer Python modules primer +=> tech/so-you-want-to-play-with-functional-programming So you want to play with functional programming + ### Gadgets => tech/gadgets/about-headphones About headphones diff --git a/programming/about_apis.md b/blog/content/notes/tech/about-apis.gmi index 2c44f41e..f1732cc9 100644 --- a/programming/about_apis.md +++ b/blog/content/notes/tech/about-apis.gmi @@ -1,6 +1,6 @@ # About APIs -The [Jeff Bezos' API memo](https://gist.github.com/kislayverma/d48b84db1ac5d737715e8319bd4dd368) is one of the most talked stories about API programming. +=> https://gist.github.com/kislayverma/d48b84db1ac5d737715e8319bd4dd368 The Jeff Bezos' API memo is one of the most talked stories about API programming. It is, in my opinion, also one of those things which are successful for some environments, but not for all. @@ -8,24 +8,19 @@ It is, in my opinion, also one of those things which are successful for some env An "operation" in your application can be in one of the following levels of "API accessibility": -* -oo The operation cannot be invoked in isolation easily. -For instance, it is embedded in an MVC controller, mixed with form handling and HTML generation, and thus the best approach to invoke it programatically is to simulate a browser +* -oo The operation cannot be invoked in isolation easily. For instance, it is embedded in an MVC controller, mixed with form handling and HTML generation, and thus the best approach to invoke it programatically is to simulate a browser * 0 The operation can be invoked, in-process, by calling a function or method, but requiring complex setup or using complex types (e.g. others than lists, maps, numbers and strings) * 1 The operation can be invoked, in-process, by calling a function without complex setup and using plain types * 2 The operation can be invoked, off-process, by calling a function without complex setup and using plain types * 3 The operation can be invoked via a command line tool * 4 The operation can be invoked via a network call -Many proponents of APIs propose level 4 as the target. -This obviously allows your operations to be integrated in separate processes via network calls, which is the most powerful way of API access. -They will also reason that this will force your application to have a clean architecture with separation of concerns. +Many proponents of APIs propose level 4 as the target. This obviously allows your operations to be integrated in separate processes via network calls, which is the most powerful way of API access. They will also reason that this will force your application to have a clean architecture with separation of concerns. Note also that doing proper testing will probably force your operations to be tested to be in levels 0-2, as otherwise it will be annoyingly complex to test them. -We propose that the architecture benefits of level 4 are also present in levels 0-3, but achieving these levels requires much less effort than achieving level 4 (where you need to add a network protocol, handle aspects such as authentication/authorization, marshalling/unmarshalling, etc.), so unless you require level 4, you can stay in levels 0-3. -Going to level 3 instead of 0 should be easy when creating new operations, so that's the level of API accessibility we recommend new code to adhere to by default. +We propose that the architecture benefits of level 4 are also present in levels 0-3, but achieving these levels requires much less effort than achieving level 4 (where you need to add a network protocol, handle aspects such as authentication/authorization, marshalling/unmarshalling, etc.), so unless you require level 4, you can stay in levels 0-3. Going to level 3 instead of 0 should be easy when creating new operations, so that's the level of API accessibility we recommend new code to adhere to by default. Note also that level 3 can provide many benefits of level 4, but with less development overhead, so it's a level we recommend considering explicitly, as it is often overlooked. -Level -oo is typical of legacy applications. -Note that we consider the distance between level -oo and the rest of levels much bigger than the distance between the rest of levels. +Level -oo is typical of legacy applications. Note that we consider the distance between level -oo and the rest of levels much bigger than the distance between the rest of levels. diff --git a/blog/content/notes/tech/about-django.gmi b/blog/content/notes/tech/about-django.gmi new file mode 100644 index 00000000..15f73600 --- /dev/null +++ b/blog/content/notes/tech/about-django.gmi @@ -0,0 +1,115 @@ +# About Django + +Without more context, one of the technologies I recommend to everyone is Django. + +Django is a Python web framework with "batteries included". + +Web frameworks can provide more or less tools to write applications. Typically, frameworks that provide fewer tools are more flexible and give developers more freedom to develop their applications in the best possible way. Similarly, frameworks that provide more tools tend to guide you towards a specific way to writing applications, and typically, require more work if you want to deviate. + +In my opinion, many applications you might need to develop are very similar and have similar issues, and solving them ad-hoc for each project is a waste. Therefore, I lean towards using frameworks that provide more batteries in most cases. + +(Certainly, there are projects that clearly need special approaches, or which deviate enough from any generic web framework.) + +In fact, most of the complaints described in this document are caused by Django having too few batteries, not too many! + +=> https://github.com/alexpdp7/django-tws Django training wheels is my project in alpha to address some of those shortcomings. + +## The Django admin + +Besides including more batteries than most other frameworks, and being in general a well-engineered framework in my opinion, Django includes the admin. + +The admin is a declarative way to build administrative sites where some users edit data stored in the application database. + +Many similar tools exist, but I have not found any other tool that can do so much. + +* The Django admin handles multi-table relationships very well, including picking foreign key targets and editing related table data. For example, if a person entity has a "parent" related foreign key relationship, the Django admin provides a search functionality to pick a person's parent. If the person entity has a list of children, the Django admin provides a way to add and edit children from the person form. +* The Django admin has a simple, but useful for many scenarios permissions functionality, where editing certain entities is restricted to groups of users. + +The Django admin is frequently a big boost during the early development of database-backed applications, and sometimes it can provide value during a big part of the life of an application. + +Additionally, traditionally when working with frameworks without an equivalent facility, the friction of adding an interface to edit a piece of data can be large. Developers pressed for time might opt to hardcode the data in the source code of the application, requiring code changes to modify certain behaviors of the application. When the friction to add a user interface to edit such data is low, developers can configure the admin to let those users edit the data directly without going through the developers. + +## Django problems + +However, there are still many common issues for which batteries could exist, but that Django does not provide. + +### Django has no support or documentation about packaging Django projects + +Most Django projects have dependencies besides Django. In order to develop and deploy Django applications, you likely must install other dependencies. Django does not include documentation nor support to do this. + +Many different approaches and tools exist to manage Python project dependencies. Understandably, endorsing one particular approach in Django could be controversial. +So Django leaves the choice of approach up to users. Additionally, Django adds a few difficulties in Python project management, and users must figure out how to handle Django projects in their chosen approach. + +Several initiatives have tried to tackle this problem, notably: + +=> https://github.com/radiac/nanodjango nanodjango + +### Django settings are a partial solution + +Django provides settings to manage the configuration for a Django project. You implement Django settings by writing a Python module. + +For example, the default Django template includes the following snippet to configure the database connection: + +``` +DATABASES = { + 'default': { + 'ENGINE': 'django.db.backends.sqlite3', + 'NAME': BASE_DIR / 'db.sqlite3', + } +} +``` + +Besides assigning a setting directly like in the preceding snippet, you can use Python code to assign settings. + +This allows you to tackle many common issues, such as setting up a different database connection for development and production, while keeping the production database credentials away from the source code repository. There are many similar issues that you must tackle in nearly all projects. + +Several initiatives tackle some of those issues: + +=> https://github.com/jazzband/dj-database-url dj-database-url provides a way to configure the database connection through an environment variable. + +### Django does not explain a development database workflow + +Django provides migrations to handle schema changes. Migrations work well and are a valid solution to handle schema changes in production. + +However, while developing a Django application, you frequently need to make many temporary changes to the data definition until you find the right data definition. + +In my opinion, if you follow the Django documentation, then you might end up using migrations for those development schema changes. This is awkward and problematic, and there are procedures to develop database changes that work better. + +I would like a command that recreates your database, applying unmigrated model changes. This command could also have hooks to load sample data. (Likely, Python code and not fixtures.) + +### Django only tackles database-based, server-side-rendered, non highly interactive web applications + +While certainly a huge amount of applications: + +* Revolve around data stored in a relational database +* Are better implemented as server-side rendering applications +* Do not require very complex or real-time interactions + +There are certainly many applications that do not fit this mold. + +In my opinion, focusing on database-based applications is a good decision. Many Django features (like the admin) revolve around the database, and a framework oriented to other applications likely should be very different. + +However, more and more applications break the limits of server-side rendering, and while you can build such applications with Django, you need a lot of effort or finding additional libraries to use. + +For example: + +* Django REST framework provides a layer to provide REST APIs on top of the Django ORM. +* Projects exist to add support for Django for front end frameworks such as htmx or Hotwire. These frameworks are an intermediate step between traditional server-side-rendered applications and JavaScript front ends, enabling most of the benefits of JavaScript front ends within the traditional server-side rendering approach. + +Additionally, providing an API is also useful beyond JavaScript front ends. APIs are necessary for other purposes, such as implementing mobile apps to interact with your application, or just providing an API for programmatic access to your application. + +### Some common tasks should have more tutorial content + +The Django documentation is mostly for reference, covering all Django features, but with little content on how to use Django. The items I list below likely are documented on books, websites, forums, etc. If you know a good source for many of those, even if it is paid, feel free to let me know to add references. + +* Restricting users in the admin to a subset of the instances of a model. + +For example, users belong to organizations and users should only see instances of some model related to their organization. The FAQ contains "How do I limit admin access so that objects can only be edited by the users who created them?", which is a very similar question and points to the features you need to use to achieve these goals. These requirements are often related to requiring "extending the existing User model". + +* Having a search UI for reference fields instead of dropdowns. + +Many projects similar to the admin only offer dropdowns for reference fields. This does not work when the referenced objects are more than a couple. Django calls this raw_id_fields, and it is difficult to learn that this feature exists. + +## Further reading + +=> crud-is-an-important-unsolved-problem CRUD is an important unsolved problem diff --git a/programming/about_relational_databases.md b/blog/content/notes/tech/about-relational-databases.gmi index 464de330..c66a530f 100644 --- a/programming/about_relational_databases.md +++ b/blog/content/notes/tech/about-relational-databases.gmi @@ -8,23 +8,20 @@ Actually, the relations in a relational database are the tables. A relation "relates" a set of values with another set of values. -For example, a relation can relate the name of a person with their birth date and birth place. -For example: +For example, a relation can relate the name of a person with their birth date and birth place. For example: -``` (person name) => (birth date, birth place) (Alice) => (1979-12-03, Barcelona) (Bob) => (1995-03-04, Paris) ... -``` Many computer languages have similar concepts: -* [Python mapping types such as `dict`](https://docs.python.org/3/library/stdtypes.html#mapping-types-dict) -* C++ `std::map` -* Java `java.util.Map` -* [C# `System.Collections.Generic.Dictionary`](https://learn.microsoft.com/en-us/dotnet/api/system.collections.generic.dictionary-2?view=net-9.0) -* [Javascript `Object`](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Object) -* [PHP arrays](https://www.php.net/manual/en/language.types.array.php) +* Python mapping types such as dict +* C++ std::map +* Java java.util.Map +* C# System.Collections.Generic.Dictionary +* Javascript Object +* PHP arrays Relations are a natural concept, so although non-relational data systems exist, most data can be stored as relations. diff --git a/programming/containers_might_not_be_the_right_answer.md b/blog/content/notes/tech/containers-might-not-be-the-right-answer.gmi index b00bbade..de614983 100644 --- a/programming/containers_might_not_be_the_right_answer.md +++ b/blog/content/notes/tech/containers-might-not-be-the-right-answer.gmi @@ -2,20 +2,17 @@ Containers are everywhere, and I feel today they are the default answer to many problems for many people. -[Although the author of one of the handiest quotes does not want people to use it](https://www.jwz.org/blog/2014/05/so-this-happened/), I think that quote adequately describes the situation. -Definitely, containers are no silver bullet. +=> https://www.jwz.org/blog/2014/05/so-this-happened/ Although the author of one of the handiest quotes does not want people to use it, I think that quote adequately describes the situation. Definitely, containers are no silver bullet. -Containers are a good example of an “easy but not simple” technique (see [the "Simple Made Easy" - Rich Hickey (2011) talk](https://www.youtube.com/watch?v=SxdOUGdseq4)). +Containers are a good example of an “easy but not simple” technique. -Containers are easy because they automate getting arbitrary "isolated" Linux environments and running processes on them. -Additionally, you can find container images for mostly everything on the Internet. -For this reason, it is very easy to run much software using containers, often with a single command. +=> https://www.youtube.com/watch?v=SxdOUGdseq4 (See the "Simple Made Easy" - Rich Hickey 2011 talk.) -However, containers are easy, but not simple. -Containers combine many different techniques to achieve their ease, and thus, frequently you hit problems derived from any of the techniques containers use. +Containers are easy because they automate getting arbitrary "isolated" Linux environments and running processes on them. Additionally, you can find container images for mostly everything on the Internet. For this reason, it is very easy to run much software using containers, often with a single command. -However, Docker popularized many good ideas, both directly related to containerization and general ideas! -There are still places where containers are the right answer. +However, containers are easy, but not simple. Containers combine many different techniques to achieve their ease, and thus, frequently you hit problems derived from any of the techniques containers use. + +However, Docker popularized many good ideas, both directly related to containerization and general ideas! There are still places where containers are the right answer. ## Reasons not to use containers @@ -35,11 +32,9 @@ Most issues can be worked around, but this requires more effort, or at least, mo ### Container images are big and expensive -Optimizing the size of container images can require significant effort. -Popular public images are often optimized for size, but even with optimized images, storing and moving container images frequently requires much more bandwidth and storage than the alternatives. +Optimizing the size of container images can require significant effort. Popular public images are often optimized for size, but even with optimized images, storing and moving container images frequently requires much more bandwidth and storage than the alternatives. -There are free ways to host private container images, but they are frequently limited in size, bandwidth, or both. -You can easily run into Docker Hub limits, GitHub only provides 2gb of storage, etc. +There are free ways to host private container images, but they are frequently limited in size, bandwidth, or both. You can easily run into Docker Hub limits, GitHub only provides 2gb of storage, etc. Building container images also can require significant resources. @@ -47,23 +42,17 @@ Building container images also can require significant resources. Using containers frequently requires learning quite a few things specific about containers. -* `Containerfile` design is not obvious. - Some questions, like `ADD` and `COPY`, or `CMD` and `ENTRYPOINT` are difficult and not well documented. +* Containerfile design is not obvious. Some questions, like ADD and COPY, or CMD and ENTRYPOINT are difficult and not well documented. -* Container design is not obvious. - Docker popularized "application containers", a fuzzy concept that is related to "single process containers", 12 factor architecture, and a few other ideas. - Solving your problem might require good knowledge of application container design and use. +* Container design is not obvious. Docker popularized "application containers", a fuzzy concept that is related to "single process containers", 12 factor architecture, and a few other ideas. Solving your problem might require good knowledge of application container design and use. -* Container tools are complex, because containerization is difficult. - Likely you need to know some intricate details of how Linux file permissions and users work, for example. +* Container tools are complex, because containerization is difficult. Likely you need to know some intricate details of how Linux file permissions and users work, for example. Not using containers can mean avoiding having to think about these things, and being able to use the time you save to actually solve your problem. ### Docker is not so good, targeting multiple container engines is not trivial -Docker was the first popular container engine. -Docker was a revolution and invented or popularized many great ideas. -However, knowledge about containers was not well established when Docker was invented, and since then, better ways of doing many things have been discovered. +Docker was the first popular container engine. Docker was a revolution and invented or popularized many great ideas. However, knowledge about containers was not well established when Docker was invented, and since then, better ways of doing many things have been discovered. Other tools such as Podman or Fedora Toolbx and adjacent tools such as Distrobox have introduced many improvements respect Docker, while still reusing and being compatible with many Docker concepts. @@ -71,8 +60,7 @@ However, creating processes and tools across these different tools can be diffic ### In some scenarios, containers do not add much -Mainly after the rise of the Go programming language, distributing binaries has become easier. -Distributing binaries on Windows and macOS has always been simpler than distributing binaries on Linux. +Mainly after the rise of the Go programming language, distributing binaries has become easier. Distributing binaries on Windows and macOS has always been simpler than distributing binaries on Linux. However, nowadays many programming languages can create binaries that can be downloaded and executed on most Linux distributions. @@ -80,8 +68,7 @@ One of the main benefits of Docker has been ease of distribution of software, bu ### Beware container images -Much software is distributed nowadays as container images. -The abundance of container images means that learning how to use containers helps you run a wide variety of software distributed as a container image. +Much software is distributed nowadays as container images. The abundance of container images means that learning how to use containers helps you run a wide variety of software distributed as a container image. However, many container images are not of great quality, nor are adequately updated. @@ -105,10 +92,9 @@ Finding the right combination that makes software portable can require significa ### Some container-related software has good and unique ideas -For example, the controversial Kubernetes still provides a distributed standardized operating system that can be managed in a declarative way. -This is a powerful concept, and still the preferred way to package software for Kubernetes depends on container images. +For example, the controversial Kubernetes still provides a distributed standardized operating system that can be managed in a declarative way. This is a powerful concept, and still the preferred way to package software for Kubernetes depends on container images. ## What to use instead of container images * Binaries -* "Portable-friendly" development tools such as Go, `uv`, or Cargo. +* "Portable-friendly" development tools such as Go, uv, or Cargo. diff --git a/programming/crud_is_an_important_unsolved_problem.md b/blog/content/notes/tech/crud-is-an-important-unsolved-problem.gmi index 88497f7e..c338f785 100644 --- a/programming/crud_is_an_important_unsolved_problem.md +++ b/blog/content/notes/tech/crud-is-an-important-unsolved-problem.gmi @@ -9,7 +9,6 @@ Although programmers have been written a huge amount of CRUD systems for decades There are two major approaches to implementing CRUD systems: * Traditional programming: combining a relational database and most existing programming languages enables programmers to create CRUD systems. - * "No code" (or "low code"): many products and services enable non-programmers to describe their data structure and the user interface requiring less technical knowledge than the traditional programming approach. ## About implementing CRUD systems with traditional programming @@ -20,24 +19,15 @@ Most of what you need to do when using Django is to describe what you need, inst Out of the box, Django provides: -* List and detail views, including nested views. - Many systems provide "flat" details view where you can edit a record, but not associated records. - For example, they provide a detail view for customers where you can edit the customer name and other information, but any "multiple" information, such as multiple addresses or phone numbers, must be edited in a different view. - This is frequently a huge issue, and it can require writing a significant amount of code in other systems. - With Django, you can implement this by describing the associated data. - -* Multi user authentication and role-based authentication. - With Django and without programming any code, administrators can create groups, assign users to groups, and limit the kinds of records that each group can view or edit. - -* Primitive change tracking. - Out of the box, changes to records are tracked automatically and can be consulted. +* List and detail views, including nested views. Many systems provide "flat" details view where you can edit a record, but not associated records. For example, they provide a detail view for customers where you can edit the customer name and other information, but any "multiple" information, such as multiple addresses or phone numbers, must be edited in a different view. This is frequently a huge issue, and it can require writing a significant amount of code in other systems. With Django, you can implement this by describing the associated data. +* Multi user authentication and role-based authentication. With Django and without programming any code, administrators can create groups, assign users to groups, and limit the kinds of records that each group can view or edit. +* Primitive change tracking. Out of the box, changes to records are tracked automatically and can be consulted. For most CRUD implementations, alternative platforms require significantly more effort to implement those features. Additionally, the entire stack is open source software that does not require paying licenses. -(Surprisingly, in the past there existed even more sophisticated CRUD platforms. - But sadly, most have disappeared.) +(Surprisingly, in the past there existed even more sophisticated CRUD platforms. But sadly, most have disappeared.) ## About implementing CRUD systems with no code @@ -51,8 +41,7 @@ However, because those systems focus on no code usage, frequently you hit roadbl When you need a feature that they do not provide, it is either impossible to do it, or it requires programming in an unfriendly environment. -Programming CRUD features can be complex. -While traditional programming tools have evolved providing many features such as automated testing and advanced code revision control systems (rolling back bad changes and others), no code CRUD platforms do not reuse the wealth of programming tools that have been developed for traditional programming. +Programming CRUD features can be complex. While traditional programming tools have evolved providing many features such as automated testing and advanced code revision control systems (rolling back bad changes and others), no code CRUD platforms do not reuse the wealth of programming tools that have been developed for traditional programming. Non-developers frequently face huge challenges going beyond the basics of what the tool provides, and developers struggle and suffer by working in environments that are more limiting compared to others. @@ -72,8 +61,7 @@ In most cases, organizations cannot justify the costs of tailoring the CRUD syst ### Improving existing traditional programming CRUD platforms -I believe systems such as Django can still see many improvements. -Likely, both the amount of technical knowledge to use these systems and the amount of effort to design CRUD systems can be reduced significantly. +I believe systems such as Django can still see many improvements. Likely, both the amount of technical knowledge to use these systems and the amount of effort to design CRUD systems can be reduced significantly. ### Providing systems to transition from no code approaches to traditional programming @@ -83,10 +71,8 @@ However, no code platforms cannot provide all features needed, and in many cases Providing a way to migrate to a traditional programming approach would enable breaking this barrier and scaling systems more effectively. -(Some no code platforms have APIs. - With them, programmers can write code to extend the no code CRUD systems using traditional programming approaches. - However, implementing functionalities through APIs has limitations and specific problems.) +(Some no code platforms have APIs. With them, programmers can write code to extend the no code CRUD systems using traditional programming approaches. However, implementing functionalities through APIs has limitations and specific problems.) ## Further reading -* [About Django](python/about_django.md). +=> about-django About Django diff --git a/blog/content/notes/tech/document-formats.gmi b/blog/content/notes/tech/document-formats.gmi new file mode 100644 index 00000000..385c0c0e --- /dev/null +++ b/blog/content/notes/tech/document-formats.gmi @@ -0,0 +1,97 @@ +# Document formats + +Most of the time, when writing a document, I want a document format with the following properties: + +* Fast to write using a plain text editor +* Easy to parse into an AST + +An AST is a programming-friendly representation of a document. ASTs reduce the effort required to write tools such as a program that validates links in a document. Ideally, ASTs contain information to track a document element to the position it occupies in the original document. With this information, if you write a tool such as a spell checker, then you can highlight misspelled works precisely in the original document. + +On top of that, some features that I don't always need: + +* Math support +* Sophisticated code blocks. For example, being able to highlight arbitrary parts of code blocks (not syntax highlighting). +* Diagram support + +## Existing formats + +### Markdown + +* Easy to write using a plain text editor +* Has good AST parsers with position information +* Has math support +* Does not support sophisticated code blocks +* There are many extensions with support for math, diagrams, and many others +* Is very popular and supported everywhere +* However, there is a wide variety of variants and quirks +* Especifically, because Markdown was not designed with parsing in mind, so tools based on different parsers can have differences in behavior + +### Djot + +=> https://djot.net + +It is very similar to Markdown, except: + +* It is designed for parsing, so independent parsing implementations are very compatible with each other +* It is not so popular, so there are less extension and tool support + +### AsciiDoc + +=> https://asciidoc.org + +Compared to Markdown: + +* It's more complex to write, but mostly because it's different and more powerful +* There are attempts to write better parsers, but good parsers with position information are not available yet +* Supports sophisticated code blocks +* It has a smaller ecosystem than Markdown, but many good quality tools such as Antora + +### Typst + +=> https://typst.app + +Checks all my boxes, except: + +* It is designed for parsing and it has an AST, but it is not easy to access +* Currently Typst is very oriented towards generating paged documents (e.g. PDF) +* It includes a full programming language, which is mostly good (very extensible), but this might increase complexity undesirably + +Typst is very new and is not yet very popular. + +=> https://codeberg.org/haydn/typesetter Typesetter is a desktop application that embeds Typst, so no additional setup is needed. However, Typesetter is only available as a Flatpak. + +### Verso + +=> https://github.com/leanprover/verso + +A Markdown-like closely tied to the Lean programming language. + +* Eliminates ambiguous syntax for easier parsing and is stricter (not all text is valid Verso) +* Has a (Lean) data model +* Designed for extensibility + +### TODO: other formats + +=> https://github.com/nota-lang/nota Nota (a document language for the browser) +=> https://github.com/christianvoigt/argdown Argdown (for argumentation) +=> https://github.com/podlite/podlite Podlite +=> https://orgmode.org Org Mode (an Emacs-based tool based on a lightweight markup language) +=> https://github.com/nvim-neorg Neorg (similar to Org Mode for Neovim) +=> https://github.com/sile-typesetter/sile Sile (typesetting system) + +## Creating your own formats + +=> https://github.com/spc476/MOPML Someone created its own lightweight format using Lua and PEGs. +=> https://tratt.net/laurie/blog/2020/which_parsing_approach.html "Which parsing approach" has information about choosing parsing approaches. + +## About gemtext + +=> https://geminiprotocol.net/docs/gemtext-specification.gmi + +Gemtext is an extremely minimalistic markup language designed for use with the Gemini protocol (an extremely minimalistic protocol similar to HTTP). + +The Gemini protocol and gemtext are intentionally designed to limit their power, in my opinion as a comment on the web. + +This document is gemtext-native for use in my own minimalistic publishing system. + +I also use it as a statement, although the limitations of gemtext can be significant in technical writing. For example, gemtext has no inline links, no inline verbatim code, only three levels of headings, etc. diff --git a/blog/content/notes/tech/git-advice.gmi b/blog/content/notes/tech/git-advice.gmi new file mode 100644 index 00000000..8ec41105 --- /dev/null +++ b/blog/content/notes/tech/git-advice.gmi @@ -0,0 +1,21 @@ +# Git advice + +## Never use "git commit -m", use "git commit -v" + +Configure your system so that the EDITOR environment variable refers to your preferred editor. + +With "git commit -v" you can see your commit diff while writing your commit message. This helps you review that your commit is correct and write a better commit message. + +## Use gitignore properly + +=> https://git-scm.com/docs/gitignore gitignore + +Note that by default, Git defaults to $XDG_CONFIG_HOME/git/ignore or $HOME/.config/git/ignore. + +## Use the modern Git commands (or teach them) + +Particularly, "git checkout" has many functionalities that now can be handled by more focused commands like "git switch" and "git reset". + +If you have too much muscle memory and are used to them, then consider learning them only to teach other people so that they start with the safer commands. + +Many Git commands print suggestions that use the newer commands. diff --git a/blog/content/notes/tech/github-annoyances.gmi b/blog/content/notes/tech/github-annoyances.gmi new file mode 100644 index 00000000..a36214b9 --- /dev/null +++ b/blog/content/notes/tech/github-annoyances.gmi @@ -0,0 +1,7 @@ +# GitHub annoyances + +## The repository creation wizard can be confusing initially + +When creating a new repo, GitHub offers you to populate the repository with some files (a README, a .gitignore file, a license). + +In some situations, you have an existing directory in your computer with files that you want to be the initial contents of the repo. If you create a truly empty repo, then GitHub displays some instructions that can help pushing the contents of your existing directory to the new repo. If you use the GitHub features to populate the repo, then GitHub does not display these instructions and uploading your files requires more knowledge. diff --git a/blog/content/notes/tech/internet-communication-channels.gmi b/blog/content/notes/tech/internet-communication-channels.gmi new file mode 100644 index 00000000..efdc18af --- /dev/null +++ b/blog/content/notes/tech/internet-communication-channels.gmi @@ -0,0 +1,131 @@ +# Internet communication channels + +If you want to provide a communication channel for a community over the Internet and you are considering options such as: + +* Slack +* Discord +* Reddit +* Telegram +* WhatsApp +* Facebook +* Or any other communication channel controlled by a single big company + +, then please read this article and consider an alternative. + +Because such channels are often convenient, cheap, and easy, they are natural choices. + +However, companies are about maximizing their benefits first. Certainly, providing convenient, cheap, and easy services often help companies make money. But I believe we have seen enough examples of companies putting their benefits first in detriment of their users. + +Using these alternatives will always require more effort. This text is long, and just reading and processing it might take more time than setting up a channel on the services mentioned above. The alternatives I describe certainly have drawbacks compared to the services I am asking you to avoid. However, in the long run I think making an extra effort to make an informed choice pays off. + +## A quick litmus test + +If you only thing about a single thing, then think about this: how many independent clients are for this communication channel? + +How tightly the people behind the channel control clients is a good indicator of how much they want to maximize profits. + +## Alternatives + +### IRC + +IRC is a real-time chat protocol created in 1988 that is still in use. Many perceive flaws in IRC that seem to make it a bad choice. However, many IRC flaws have been addressed in recent times and I believe it is a good choice in many (but not all) scenarios. + +The biggest traditional issue with IRC is channels without history, where you cannot see messages posted while you were offline. (If you suspend or turn off your laptop, you will be offline in IRC. Even if you run your IRC client continuously on your client, if your phone goes out of coverage or your phone suspends your IRC client, you will be offline.) However, nowadays you can create channels with history. + +Channels without history are frequently confusing for new users, because most chat systems have history. Heavy IRC users are either used to having no history [this might seem surprising, but for some this is even a benefit] or have means to be permanently connected to IRC. However, users new to IRC might join a channel, post a question and go offline without anyone having a chance to see their message and reply. Then, unless people remember to answer when they are back, or other means are used, answers will not be visible to the person who asked. + +The chathistory extension addresses this problem. As far as I know, only the Ergo IRC server and their network support this extension. + +Some advantages of IRC are: + +* You can use IRC without creating an account. This can be especially useful for providing a general contact mechanism. You can create links that will ask for a nickname, and place you into a channel without any additional steps. +* IRC is a very simple protocol with more than 30 years of history. This means that many developers have invested significant efforts in creating powerful IRC clients and tools (such as bots). And lately, many easy IRC clients are available. This means that IRC can scale from simple setups that require little effort to use, to powerful setups that can provide interesting features. (If you are used to plain communication clients, you might be surprised at how valuable some features can be.) + +Some drawbacks of IRC are: + +* IRC does not have end-to-end encryption, and thus IRC administrators can read every conversation. This is not a huge problem for public or semi-public channels, but it limits IRC for many scenarios. +* IRC requires more effort from administrators to provide a good experience to entry-level users, control spam, and others. (An important point is that although integration with audio/video conferencing is possible, it requires more effort and provides a lesser experience.) +* IRC is mostly text-based. Although many IRC clients can display images and GIFs, communicating with images and GIFs is harder on IRC. (And IRC also does not have integrated audio/video conferencing.) +* Push notifications are not common yet. Although it is possible to receive instant notifications when you are mentioned or receive a private message, this is frequently difficult. In general, IRC on mobile phones is not as evolved as on desktop computers. + +Interesting projects: + +=> https://github.com/ObsidianIRC/ObsidianIRC ObsidianIRC (web client) +=> https://kiwiirc.com/ Kiwi IRC (web client) +=> https://codeberg.org/emersion/gamja Gamja (web client) +=> https://soju.im/ Soju (bouncer) +=> https://halloy.chat/ Halloy (graphical desktop client) +=> https://git.sr.ht/~delthas/senpai Senpai (terminal client) + +### Delta Chat + +Delta Chat is an instant messaging system that tries to be very similar to the most popular instant messaging systems. + +However, there are multiple clients and anyone can run a server. + +The strangest thing about Delta Chat is that is uses email underneath. However, I would recommend ignoring this fact. + +### XMPP + +XMPP is younger than IRC, but older than Matrix. Compared to Matrix: + +* End-to-end encryption and audio/video conferencing is possible with XMPP, but in practice it can be difficult to access these features. +* There's more XMPP clients than Matrix clients, but it is also hard to find clients that support all the features you need on different platforms. + +For some scenarios, if you find the right combination of XMPP server and clients, XMPP can be a great option. + +Historically, XMPP was not well-suited to mobile usage. Nowadays, mobile usage is better, but finding the right clients to use is still a challenge. + +### Matrix + +Matrix is a more modern chat protocol that addresses some of the drawbacks of IRC: + +* Matrix has end-to-end encryption, so conversations between users are private to Matrix administrators. +* Matrix requires less effort from *channel* administrators. (But running a Matrix server requires significant resources. However, there are public Matrix servers and managed services. Thanks to end-to-end encryption, using a public Matrix server is an interesting option.) +* Matrix has good support for audio/video conferencing, images and GIFs, reactions, push notifications, and phone usage. + +But also some disadvantages compared to IRC: + +* Users need to create accounts. +* Using end-to-end encryption makes some usage harder. (Although end-to-end encryption is optional.) +* There are fewer clients and tools, and generally they are more complex, more resource intensive, and less featureful. (And not all clients support all features.) + +### Other alternatives to consider + +Zulip offers instant messaging, but has some characteristics from forums. (For example, Zulip uses threads with subjects.) + +Mattermost and Rocketchat are designed for communication within organizations. + +And lastly, because all the technologies mentioned in this text allow integrations, there are bridges to join different technologies. + +For example, IRC channels can be bridged to Matrix rooms. + +Although bridges are not ideal, in some cases you can use them to make one channel available over different technologies, which might address the limitations of specific technologies. + +Although my perception is that most communities nowadays communicate over instant messaging, many communities use successfully more asynchronous communication channels. In some cases, providing both instant messaging and an asynchronous channel can also work well. + +Mailing lists (and their sibling, newsgroups) are older than IRC. Although mailing lists are far less popular than in the past, many communities still use mailing lists. + +Mailing lists have several advantages: + +* Having an email address is nearly a necessity for all Internet users. Mailing lists often require no user account other than an existing email address. +* In a way, email and mailing lists share many similarities with IRC. Although most people are users of just a few mail services and clients, there is a wide variety of services and clients. Email power features are somewhat forgotten, but they still exist and mail clients can have very convenient features. +* Most mailing list have good ways to browse and search past messages. Email discussions are more naturally searchable, thanks to their slower pace and thread organization. + +However, they also have many advantages: + +* As people no longer use email to communicate, going back to email can cause significant friction. +* Finding a good mailing list service is difficult. (And hosting your own is also more difficult than hosting other services.) + +In my opinion, mailing lists are good, but they have become foreign to most people. + +Forums used to be very popular. + +Compared to mailing lists: + +* Forums require creating an account. +* Forums do not have multiple clients, although forum software has also evolved for a long time, and many forums have great features. +* Forums are also a bit out of style, but they are more popular and familiar to most than mailing lists. +* Finding a forum service or hosting one is simpler than email. + +Social networks tend to be slightly different communication channels than instant messaging or asynchronous messaging. Alternatives to social networks also exist. However, in my opinion, social network-style communication is not optimal for "communities" in most cases. Still, you might want to explore alternatives. The Fediverse (or ActivityPub) has many different varieties of communication channels that might suit your needs. diff --git a/misc/ledger.md b/blog/content/notes/tech/ledger.gmi index 18c6f1f1..97a0af3f 100644 --- a/misc/ledger.md +++ b/blog/content/notes/tech/ledger.gmi @@ -1,13 +1,14 @@ # ledger -[ledger](https://ledger-cli.org/) is a double-entry accounting system based on a text file format. -The [Plain text accounting](https://plaintextaccounting.org/) website lists more software based on the ideas. +=> https://ledger-cli.org Ledger is a double-entry accounting system based on a text file format. + +=> https://plaintextaccounting.org/ The Plain text accounting website lists more software based on the ideas. This document contains notes about how I use ledger. ## Configuration -My `~/.ledgerrc` just contains: +My ~/.ledgerrc just contains: ``` --file ~/Nextcloud/finances.ledger @@ -26,8 +27,7 @@ I do so manually without automations. In 2025, I registered over 800 transactions, and I didn't feel it was tedious. -My main text editor is Emacs, so I use [ledger-mode](https://github.com/ledger/ledger-mode/). -ledger-mode: +My main text editor is Emacs, so I use ledger-mode: * Automatically adds indentation and alignment. * Autocompletion of accounts and payees. @@ -40,20 +40,18 @@ ledger reg bankname:accountname Then, I correlate with the running total that my bank websites show to find the first missing transaction and go on from there. -I have a monthly calendar reminder to catch up on all accounts. -In this session, I also update my pension plan accounts with their current value. +I have a monthly calendar reminder to catch up on all accounts. In this session, I also update my pension plan accounts with their current value. ## Tagging -### `who` +### who -I use the `who` tag because I want to make reports based on specific beings. -For example, I want to query quickly costs associated to the cat. +I use the who tag because I want to make reports based on specific beings. For example, I want to query quickly costs associated to the cat. -In 2026, I think I will have some accounts like `Expenses:Supermarket:My Name` too, so I'm experimenting with the following snippet: +In 2026, I think I will have some accounts like "Expenses:Supermarket:My Name" too, so I'm experimenting with the following snippet: ``` -= My Name += :My Name ; who: myname ``` diff --git a/misc/migadu.md b/blog/content/notes/tech/migadu.gmi index b4abc992..9fc5a80e 100644 --- a/misc/migadu.md +++ b/blog/content/notes/tech/migadu.gmi @@ -1,44 +1,41 @@ # Migadu -<https://migadu.com/> +=> https://migadu.com/ ## A strategy to handle email addresses Email addresses can be "vanity" or "non-vanity". -Vanity email addresses are meant to be public and memorable, for example `firstname@lastname.tld`. +Vanity email addresses are meant to be public and memorable, for example firstname@lastname.tld. Set up any vanity domains that you want, and a non-vanity domain. ### Non-vanity email addresses -In the non-vanity domain, you create the `{me}{code}@nonvanity.tld` mailbox. +In the non-vanity domain, you create the {me}{code}@nonvanity.tld mailbox. -`{me}` identifies you, you can have multiple users with different `{me}` identifiers in a single non-vanity domain. +{me} identifies you, you can have multiple users with different {me} identifiers in a single non-vanity domain. -This strategy uses `{code}` to obfuscate email addresses. -When you use `{code}` in an email address, you should be able to identify if the `{code}` is valid or not. +This strategy uses {code} to obfuscate email addresses. When you use {code} in an email address, ideally you should be able to identify if the {code} is valid or not. -For example, you could use a four-digit `{code}` and store what code you have used for each address. -If you use `x3452` and store this code, when you receive an email that does not match, such as `x3453`, you know the code is incorrect. +For example, you could use a four-digit {code} and store what code you have used for each address. If you use x3452 and store this code, when you receive an email that does not match, such as x3453, you know the code is incorrect. Alternatively, you can use hashing so that you do not have to store all codes. -No one except you should know about `{me}{code}@nonvanity.tld`. +No one except you should know about {me}{code}@nonvanity.tld. -Then you create a pattern rewrite from `{me}.*@nonvanity.tld` to `{me}{code}@nonvanity.tld`. +Then you create a pattern rewrite from {me}.*@nonvanity.tld to {me}{code}@nonvanity.tld. -When you need a non-vanity email address, you create a new `{me}.{entity}{code}@nonvanity.tld`, where `{entity}` is the entity that communicates with this email address and `{code}` is a **new** code. +When you need a non-vanity email address, you create a new {me}.{entity}{code}@nonvanity.tld, where {entity} is the entity that communicates with this email address and {code} is a new code. -Mails received at `{me}@nonvanity.tld` are incorrect. -Mails received without the correct code are incorrect. +Mails received at {me}@nonvanity.tld are incorrect. Mails received without the correct code are incorrect. ### Vanity email addresses -Create any needed `{id}@vanity.tld` addresses. +Create any needed {id}@vanity.tld addresses. Different from non-vanity email addresses, vanity email addresses can be guessed and you cannot identify invalid email. -See [email forwarding via IMAP](../linux/misc.md#email-forwarding-via-imap) for notes about forwarding between different email servers. +=> misc-linux-stuff See "email forwarding via IMAP" for notes about forwarding between different email servers. ### TODO Filing @@ -50,18 +47,15 @@ Because each vanity email address and entity has a different email address, you imapsync --user1 xxx@gmail.com -passfile1 gmailpass --user2 a@a.com --host2 imap.a.com --passfile2 pass --gmail1 ``` -To move mail, add `--delete1`. -But this seems to make `imapsync` much slower! +To move mail, add --delete1. But this seems to make `imapsync` much slower! (IIRC, also this didn't remove the emails from GMail!) ### Preventing issues with multiple tags -An email message can have multiple "tags" in Gmail that correspond to IMAP folders. -If you have messages with multiple tags, then the migration will duplicate messages in multiple folders or file mails to one folder at "random". +An email message can have multiple "tags" in Gmail that correspond to IMAP folders. If you have messages with multiple tags, then the migration will duplicate messages in multiple folders or file mails to one folder at "random". imapsync has features to control this, and avoid problems with the "all mail" and "sent mail" Gmail folders, but for further control, you can refile emails to have a single tag. -I have an mbsync replica of my Gmail account for backup purposes. -This replica can be used to find messages with multiple tags: +I have an mbsync replica of my Gmail account for backup purposes. This replica can be used to find messages with multiple tags: ``` find . -path './\[Gmail\]/All Mail' -prune -o -not -name index -type f -exec grep -H ^Message-ID: {} \; >index diff --git a/blog/content/notes/tech/misc-linux-stuff.gmi b/blog/content/notes/tech/misc-linux-stuff.gmi new file mode 100644 index 00000000..a5521f1f --- /dev/null +++ b/blog/content/notes/tech/misc-linux-stuff.gmi @@ -0,0 +1,116 @@ +# Misc Linux stuff + +## Reverse sshfs + +=> https://blog.dhampir.no/content/reverse-sshfs-mounts-fs-push + +You need the SFTP server program on your local machine (on Debian, the openssh-server package) and sshfs on the remote machine. + +## Find non-Debian packages + +``` +aptitude search '~S ~i !~ODebian' +``` + +## Memory usage queries + +### systemd + +``` +systemd-cgtop -m +``` + +Drill down with: + +``` +systemd-cgtop -m user.slice/user-1000.slice +``` + +### smem + +``` +sudo smem -P beam.smp -kta +``` + +## Quick rerouting of browser traffic through another host + +"ssh -D 1234 host" creates a Socks proxy on localhost:1234 that sends traffic through host. + +By enabling "allow extension to control proxy settings" in the multi account containers Firefox add-on, you can make containers use specific proxies. + +## Email forwarding via IMAP + +When you have multiple email addresses, you have multiple options to use them. + +Traditionally, people used redirection/forwarding to make email arriving at from@example.com go to to@example.net instead. + +If mail from example.com and example.net is handled by different servers, typically you can configure the example.com mail server to resend any message arriving to the from address to to@example.net. + +However, nowadays with spam filtering, the example.net mail server can reject these emails as spam, sometimes silently. + +For safer redirects, you can: + +* Use the same mail server for both accounts. However, this sometimes cannot be done or has extra cost and complexity. + +* Configure the destination email server to fetch email from the origin mail server. For example, Gmail can do this, but the fetching period can be as long as 15 minutes. This can be inconvenient when receiving confirmation emails, etc. Additionally, operators of the destination email server now have your credentials. + +A third option is to run this fetching process yourself. + +=> https://github.com/lefcha/imapfilter IMAPFilter supports connecting to an IMAP account, waiting until messages to arrive, and moving them to another IMAP account. + +Benefits: + +* IMAPFilter can use IMAP idle to request the IMAP server to notify when messages arriving, so forwarding happens without a polling delay. +* Because IMAP is used on both sides, no spam filtering happens. +* IMAPFilter is packaged for many distributions. + +Drawbacks: + +* Requires additional infrastructure. +* If IMAPFilter stops working, email stops being forwarded without warning. + +=> https://ñix.es/cgit/alex/alexpdp7.git/tree/scripts/p7s/mail/__init__.py?h=master Refer to this Python module for scripts that configure IMAPFilter as a systemd service, with credentials from Bitwarden. + +## Using a PCSensor pedal + +=> https://github.com/rgerganov/footswitch Install Footswitch + +``` +sudo footswitch -1 -m ctrl -2 -S '7F' -3 -m win -k m +``` + +Maps: + +* Left pedal to be a ctrl key +* Center pedal to the toggle mute key +* Right pedal to be Win+M (you can map this in Gnome to toggle mic status) + +## Remote networking + +If you can create a pipe between two hosts (using SSH, for example), you can use VDE (Virtual Distributed Ethernet) to connect the two hosts over a virtual network. + +You need the following programs on both hosts: + +* dpipe and vde_plug (on Debian, use the vdeplug package) +* vde-switch (on Debian, use the vde-switch package) + +Run as root on both hosts: + +``` +vde_switch -t tap0 +``` + +This command creates a virtual switch connected to tap0. + +Use the dpipe command to connect two instances of the vde_plug command running as root on both hosts. + +``` +$ dpipe sudo vde_plug = ssh root@remote vde_plug +``` + +Then bring the tap0 interface up and configure IP addresses on both hosts. + +``` +# ip link set tap0 up +# ip addr add $IP/$NETMASK dev tap0 +``` diff --git a/blog/content/notes/tech/misc-python-stuff.gmi b/blog/content/notes/tech/misc-python-stuff.gmi new file mode 100644 index 00000000..6672e021 --- /dev/null +++ b/blog/content/notes/tech/misc-python-stuff.gmi @@ -0,0 +1,62 @@ +# Misc Python stuff + +## Tools + +* Use uv for your software. It's modern and good. +* pipx is packaged by many distros and useful for installing Python software. Provide instructions for installing your software using pipx. +* Use pytest for testing. It's PEP-8 compliant unlike unittest in the standard library. doctest is good too. +* Use ruff for validation. + +## Python versions + +Try to support Python versions in popular LTS distros such as RHEL and its clones, and Debian and its derivates (like Ubuntu). + +## Writing command line tools + +* Use entry points so that when using pipx or uv to install your tool, they install the tool to the user's path. + +### Nice Python libraries + +* appdirs for using the proper configuration, cache, etc. directories on multiple platforms +* keyring for cross-platform secret storage +* tqdm for progress bars + +### Writing dependencyless Python + +If you can write your program without using dependencies, then it automatically becomes much easier to distribute and run. This is quite possible for many tools! And helps you replace problematic shell scripts. + +The standard library still includes a lot of batteries: + +* argparse is clunkier that third-party libraries, but it works well enough. The documentation provides a pattern for subcommands under "one particularly effective way of handling sub-commands...". argcomplete is a third party library that adds tab completion for argparse programs. Review the types that argparse provides support for, such as Path, enumerated choices, etc. +* compression allows working with many archive formats without shelling out to tar or (un)zip. +* concurrent.futures for executing things in parallel, esp. the map function combined with a ThreadPoolExecutor (for IO-bound tasks) or ProcessPoolExecutor (to avoid the GIL in tasks that use the CPU). +* getpass to read passwords from the terminal properly. Also to obtain the current user. +* hashlib to avoid shelling out to commands such as sha256sum. +* http.server is useful for simple web servers (and also as a quick web server in the command line). +* json is about the only structured format supported in the standard library. +* logging to print output with timestamps. +* pathlib for any kind of path manipulation, esp. the read|write_text|bytes methods that are so convenient. shutil still contains a few functions missing from pathlib, esp. in older Python versions. +* textwrap.dedent and str.[lr]strip for embedding multiline strings in code. +* urllib.request is clunkier than third-party libraries, but it's usable. + +(Also for very simple stuff, tkinter can implement simple graphical tools.) + +### Subprocess + +The main problem of using Python for scripting is that the subprocess functions do not default to check=True and shell=False. + +Likely many of your scripts will start with a subprocess wrapper suited to your script. + +You can use shlex.join to print commands you execute in a copy-pastable way. + +## Writing scrapers + +=> https://playwright.dev/python/ Use Playwright + +* Playwright automatically sets up headless browsers. +* Provides convenient abstractions for locating elements in a page (mostly no XPath required. It can match "intelligently" using text). +* Has a handy UI tool that records your actions in a browser and writes equivalent *readable* Playwright code. + +Further reading: + +=> https://new.pythonforengineers.com/blog/web-automation-dont-use-selenium-use-playwright/ Web automation: don't use Selenium, use Playwright diff --git a/programming/prolog_vs_sql.md b/blog/content/notes/tech/prolog-vs-sql.gmi index beb92c01..181ae7e3 100644 --- a/programming/prolog_vs_sql.md +++ b/blog/content/notes/tech/prolog-vs-sql.gmi @@ -1,15 +1,12 @@ # Showing the similarities between SQL and Prolog -SQL is a very common programming language, which sometimes is compared to the relatively more obscure Prolog language. -Both are examples of declarative languages, where you define some facts, then you can ask questions about those facts, and the system answers the questions without you writing an explicit program. +SQL is a very common programming language, which sometimes is compared to the relatively more obscure Prolog language. Both are examples of declarative languages, where you define some facts, then you can ask questions about those facts, and the system answers the questions without you writing an explicit program. -However, I could not find a good example of their similarities. -This text presents the most typical Prolog example, and translates it to SQL. +However, I could not find a good example of their similarities. This text presents the most typical Prolog example, and translates it to SQL. ## A typical Prolog example -`[x]` reads Prolog facts from file `x`. -We can use the special file `user` to read facts from the REPL, ending the facts with ctrl+d: +"[x]" reads Prolog facts from file "x". We can use the special file "user" to read facts from the REPL, ending the facts with ctrl+d: ``` @@ -23,8 +20,7 @@ $ swipl true. ``` -You should read `father(X,Y)` as "`X` is the father of `Y`". -So Jim is the father of Julian, and so on. +You should read "father(X,Y)" as "X is the father of Y". So Jim is the father of Julian, and so on. We can ask Prolog questions: @@ -33,8 +29,7 @@ We can ask Prolog questions: false. ``` -Is Julian the father of Jim? There is no known fact about this, so no. -But Julian *is* the father of Joe: +Is Julian the father of Jim? There is no known fact about this, so no. But Julian *is* the father of Joe: ``` ?- father(julian, joe). @@ -49,7 +44,7 @@ X = joe ; X = jerome. ``` -(You press `;` to get further answers.) +(You press ; to get further answers.) ## A simple translation to SQL @@ -86,8 +81,7 @@ The typical example continues with some logic: true. ``` -`X` is the grandfather of `Y` if `X` is the father of `Z`, and `Z` is the father of `Y`. -Then you can ask questions, and Prolog knows the answers: +X is the grandfather of Y if X is the father of Z and Z is the father of Y. Then you can ask questions, and Prolog knows the answers: ``` ?- grandfather(jim, X). diff --git a/blog/content/notes/tech/python-modules-primer.gmi b/blog/content/notes/tech/python-modules-primer.gmi new file mode 100644 index 00000000..67d03e19 --- /dev/null +++ b/blog/content/notes/tech/python-modules-primer.gmi @@ -0,0 +1,229 @@ +# Python Modules Primer + +## Prerequisites + +These instructions assume a Linux environment. A macOS environment is similar, but not identical. A Windows environment is more different. + +## Previous knowledge + +### A refresher on the PATH variable + +If you execute the following command in your terminal: + +``` +$ echo hello +``` + +, the shell searches for the echo command in the directories listed in your PATH environment variable. You can display your PATH variable by running: + +``` +$ echo $PATH +/home/user/.local/bin:/home/user/bin:/usr/share/Modules/bin:/usr/local/bin:/usr/local/sbin:/usr/bin:/usr/sbin +``` + +The contents of the PATH variable depend on your particular environment. + +If you run the following command: + +``` +$ which echo +/usr/bin/echo +``` + +The which command prints where the shell locates the echo command. + +### A refresher on shell scripts + +If you create a file named foo.sh with the following contents: + +``` +#!/bin/sh + +echo hello +``` + +, you define a "shell script". The first line indicates that this shell script is executed by using the /bin/sh command. The rest of the file are commands to be executed by the shell command. These commands behave as if you typed them into your terminal, so if you execute this script, the command "echo hello" will be executed, printing hello. + +If you try to run foo.sh like you run the echo command, by typing its name, it does not work: + +``` +$ foo.sh +bash: foo.sh: command not found... +``` + +, because the shell looks for the foo.sh in the directories listed in the PATH variable. Unless you created the foo.sh file in a directory like /usr/bin, the shell will not find the foo.sh command. + +A solution to this problem is to specify the path to the foo.sh file, instead of relying on the PATH variable. However, if you do this, you face a second problem. + +``` +$ ./foo.sh +bash: ./foo.sh: Permission denied +``` + +This happens because only files with the executable permission can be executed in this way. To solve this, add the executable permission; then it works: + +``` +$ chmod +x foo.sh +$ ./foo.sh +hello +``` + +## The import statement in Python + +### Importing from the Python standard library + +Run the following commands by using the Python REPL: + +``` +$ python3 +>>> import datetime +>>> datetime.datetime.now() +datetime.datetime(2023, 9, 11, 21, 53, 16, 331236) +``` + +import works in a similar way to running a command in the shell. +Python searches a number of directories looking for the datetime module. + +To see which directories are searched, run: + +``` +$ python3 +>>> import sys +>>> sys.path +['', '/usr/lib64/python39.zip', '/usr/lib64/python3.9', '/usr/lib64/python3.9/lib-dynload', '/home/alex/.local/lib/python3.9/site-packages', '/usr/lib64/python3.9/site-packages', '/usr/lib/python3.9/site-packages'] +``` + +sys.path is a list of the directories that the import command searches. +The contents of `sys.path` depend on your operating system and Python installation method. + +In my system, the /usr/lib64/python3.9 directory contains the datetime.py module. + +``` +$ head /usr/lib64/python3.9/datetime.py +"""Concrete date/time and related types. + +See http://www.iana.org/time-zones/repository/tz-link.html for +time zone and DST data sources. +""" + +__all__ = ("date", "datetime", "time", "timedelta", "timezone", "tzinfo", + "MINYEAR", "MAXYEAR") +... +``` + +/usr/lib64/python3.9 contains the modules in the Python standard library. + +### Importing your Python files + +If you create a file with the a.py name: + +``` +def f(): + return 2 +``` + +, and another with the b.py name: + +``` +import a + +print(a.f()) +``` + +, then: + +``` +$ python b.py +2 +``` + + +This works, because sys.path contains '', which means "the current directory". + +(sys.path is very similar to the PATH variable. However, sys.path contains the current directory by default, whereas PATH does not.) + +When "import a" is executed, then Python searches the directories in sys.path for an a.py file; it is found when checking the '' path. When "import datetime" is executed, Python searches in the current directory (because '' comes first in the path), doesn't find it, but then finds it in the following /usr/lib64/python3.9 directory. Python iterates over the sys.path directories, and loads the first matching file. + +## Installing libraries + +When writing Python software, sometimes it is enough with the modules included in the standard library. However, frequently you want to use other libraries. To use Python libraries, you must install them using the pip program. + +The pip program is not part of the python3 package in some Linux distributions, and comes from the python3-pip package. + +The pip program can download libraries from pypi.org, the Python package index, and install them. pip installs libraries to a "Python environment". + +Old versions of pip defaulted to installing libraries to the "system" Python environment. In a Linux system, the system Python environment is located in a directory such as /usr/lib64/python3.9. By default, normal Linux users cannot write to /usr, so installing a package would fail. + +Modern versions of pip detect that they cannot write to the "system" Python environment, and then redirect the install to the "user" Python environment. The "user" Python environment is in a directory such as ~/.local/lib/python3.9. + +You could use a command such as "sudo pip install" to grant pip the privileges required to write to /usr. However, this can make a Linux system unusable. Most Linux systems use software that uses the "system" Python environment. Altering the "system" Python environment can break such software. Do not run "sudo pip install" with root privileges unless you know why you need this. + +If you use a modern pip (or use the --user option), you can install libraries to the "user" Python environment. However, this is problematic because a Python environment can only contain a single version of a Python library. If you have two different Python programs that different versions of the same library, then these two programs cannot coexist in the "user" Python environment. + +In general, Python virtual environments are used to address this problem. + +## Creating Python virtual environments + +If you run: + +``` +$ python3 -m venv <some path> +``` + +This will create a directory with the path you specify, with the following contents: + +``` +<some path> +├── bin +│ ├── activate +│ ├── pip +│ ├── python +├── include +├── lib +│ └── python3.9 +``` + +The python and pip commands are copies of the same commands from the "system" Python environment. + +But these commands work differently from the "system" Python environment commands: + +``` +$ <some path>/bin/python +>>> import sys +>>> sys.path +['', '/usr/lib64/python39.zip', '/usr/lib64/python3.9', '/usr/lib64/python3.9/lib-dynload', '<some path>/lib64/python3.9/site-packages', '<some path>/lib/python3.9/site-packages'] +``` + +sys.path uses the lib directories in the virtual environment. + +When you use the `pip` program from the virtual environment, it installs the libraries to the virtual environment. + +You can create as many virtual environments as you need, and you can install different versions of libraries to each virtual environment. + +## Activating Python environments + +You can run the python and pip commands by specifying the full path, like we did when executing the foo.sh command earlier. + +By default, if you run python, the shell will invoke the python command from the "system" Python environment because it is in a directory included in the PATH variable. If you specify the full path, you override this. + +To save typing, the bin directory of a virtual environment contains an activate file. The activate file is a "special" shell script that must be invoked like this: + +``` +$ . <some path>/bin/activate +``` + +. is a special shell command that the activate script requires to work correctly. + +activate alters your path, so that the bin directory in your virtual environment comes first in your path. + +``` +$ echo $PATH +/home/user/.local/bin:/home/user/bin:/usr/share/Modules/bin:/usr/local/bin:/usr/local/sbin:/usr/bin:/usr/sbin +$ . <some path>/bin/activate +(some path) $ echo $PATH +<some path>/bin:/home/user/.local/bin:/home/user/bin:/usr/share/Modules/bin:/usr/local/bin:/usr/local/sbin:/usr/bin:/usr/sbin +``` + +, and thus if you run python, <some path>/bin/python will be executed instead of /usr/bin/python. + +Besides changing your prompt to indicate the virtual environment is activated, activate only alters your PATH. activate is not mandatory to use a virtual environment. For example, when running the Python command, if you specify the path of the Python executable in a virtual environment, the command will execute as if the virtual environment had been activated. Virtual environment management tools also have commands that can run commands inside a virtual environment without activating it. Activation can save time, but it is also more error-prone than more explicit means of using virtual environments. diff --git a/blog/content/notes/tech/ripping.gmi b/blog/content/notes/tech/ripping.gmi new file mode 100644 index 00000000..c2ab2ede --- /dev/null +++ b/blog/content/notes/tech/ripping.gmi @@ -0,0 +1,67 @@ +# Ripping + +## Media + +=> https://arstechnica.com/civis/threads/ripping-optical-media.1507399/post-43734994 Main source + +### Audio CD + +About 200-300 MB per album CD when ripped to FLAC. + +### DVD + +About 4-8 GB per disc, averaging 5.6 GB per movie as ISO. + +### Blu-ray + +About 20-50 GB per disc, averaging 37 GB per movie as ISO. + +## Hardware + +### Reader + +I got a Verbatim external USB Blu-ray writer for about 120€. + +### Storage + +=> https://diskprices.com/ See diskprices.com + +## Software + +### Audio + +=> https://abcde.einval.com/wiki/ abcde claims to rip and compress to FLAC and tag automatically. + +### DVD + +Use dd to rip DVD. However, dd can fail on some disks, perhaps due to damage or copy protection. + +=> https://unix.stackexchange.com/a/642790 This post on unix.stackexchange describes a trick that works: + +* Start playback of the disc using VLC. +* Try dd first, if it fails, then run a command like "ddrescue -n -b2048 -K1M /dev/sr0 x.iso x.map". +* After dd or ddrescue starts running, quit VLC. + +For playback, most software (including Kodi and VLC for Android) can play back DVD ISO with full menu support + +### Blu-ray + +=> http://fvonline-db.bplaced.net/ FindVUK has the keys to play Blu-ray discs ripped with dd. + +However, with encrypted Blu-ray discs, you need to configure the keys in each device where you want to play back the content. (And this is not easy or possible in some cases.) + +=> https://git.sr.ht/~shironeko/blu-save blu-save can remove the encryption. + +Remember to specify the path to the keys when running blu-save. + +However, VLC is confused by the AACS and CERTIFICATE directories that blu-save copies to the output. If you remove them, then VLC can play the BDMV directory with menus, etc. + +You can repack a Blu-ray extracted with blu-save by running a command like: + +``` +mkisofs -allow-limited-size -o .../my.iso . +``` + +from the directory that contains *only* the BDMV directory. + +VLC for desktop computers can open a repacked Blu-ray ISO and show the menus. Kodi for Android can open a repacked Blu-ray ISO and identify the titles. However, Kodi did not support the menus for the Blu-ray I tested. diff --git a/blog/content/notes/tech/running-commands-in-linux.gmi b/blog/content/notes/tech/running-commands-in-linux.gmi new file mode 100644 index 00000000..4fe4a004 --- /dev/null +++ b/blog/content/notes/tech/running-commands-in-linux.gmi @@ -0,0 +1,259 @@ +# Running commands in Linux + +## Motivating examples + +=> https://cwe.mitre.org/data/definitions/1337.html The 2021 CWE Top 25 Most Dangerous Software Weaknesses helps focus on the biggest security issues that developers face. + +=> https://cwe.mitre.org/data/definitions/78.html Number 5 on that list is Improper Neutralization of Special Elements used in an OS Command ('OS Command Injection'). + +Software developers often write code that invokes other programs. For example, shell scripts tend to be mostly composed of invocations of programs such as find, grep, etc. Even software developed in languages such as Python, C, or Java often invokes other programs. + +Python software developers use the subprocess module to perform this task. Other languages provide similar facilities. + +Consider the two following Python sessions to execute an equivalent to the bash statement "cat /etc/passwd": + +``` +$ python3 +>>> import subprocess +>>> subprocess.run(["cat", "/etc/passwd"]) +``` + +``` +$ python3 +>>> import subprocess +>>> subprocess.run("cat /etc/passwd", shell=True) +``` + +Both scripts use the same run function, with different values of the shell parameter (the shell parameter defaults to True). When executing a command with many arguments, shell=True seems to be terser. "a b c d e" is shorter and easier to read than ["a", "b", "c", "d", "e"]. Readable code is easier to maintain, so a software developer could prefer the shell=True version. + +However, using shell=True can introduce the "OS Command Injection" weakness easily. + +Create a file named "injection.py" with the following contents: + +``` +import sys +import subprocess + +subprocess.run(f"cat {sys.argv[1]}", shell=True) +``` + +This program uses the cat command to display the contents of a file. +For example, if you run (using Python 3.6 or higher): + +``` +$ python3 injection.py /etc/passwd +``` + +The terminal shows the contents of the `/etc/passwd` file. + +However, if you run: + +``` +$ python3 injection.py '/etc/passwd ; touch injected' +``` + +The terminal shows the same file, but a file named `injected` also appears in the current directory. + +Create a file named "safe.py" with the following contents: + +``` +import sys +import subprocess + +subprocess.run(["cat", sys.argv[1]]) +``` + +Running "python3 safe.py /etc/passwd" has the same behavior as using injection.py. However, repeating the command that creates a file using safe.py results in: + +``` +$ python3 safe.py '/etc/passwd ; touch injected' +cat: '/etc/passwd ; touch injected': No such file or directory +``` + +injection.py is vulnerable to "OS Command Injection" because it uses shell=True, whereas safe.py is not. + +If a malicious user can get strings such as "/etc/passwd ; touch injected" to code that uses shell=True, then the user can execute arbitrary code in the system. Code that does not handle user input might not be exposed to such issues, but user input might creep in and introduce unexpected vulnerabilities. Avoiding the use of `shell=True` and similar features can be safer than making sure that user input is correctly handled in all cases. + +## Writing shell scripts that handle files with spaces in their names + +Create a file called backup.sh with the following contents: + +``` +#!/bin/bash + +for a in $1/* ; do + cp $a $a.bak +done +``` + +Run the following statements in the terminal to create a sample directory with files. + +``` +$ mkdir backup_example_1 +$ for a in $(seq 1 9) ; do echo $a >backup_example_1/$a ; done +``` + +These statements create the backup_example_1 directory, and files named 1 ... 9. + +The backup.sh script creates a copy of each file in a directory. If you run: + +``` +$ bash backup.sh backup_example_1/ +``` + +Then the script will copy 1 to 1.bak, and so on. + +However, if you create a new directory with files whose names have spaces: + +``` +$ mkdir backup_example_2 +$ for a in $(seq 1 9) ; do echo $a >backup_example_1/"file $a" ; done +``` + +Then the backup.sh script does not work correctly: + +``` +$ bash backup.sh backup_example_2/ +cp: cannot stat 'backup_example_2//*': No such file or directory +``` + +In order to fix the script, change the contents of backup.sh to: + +``` +#!/bin/bash + +for a in "$1/*" ; do + cp "$a" "$a.bak" +done +``` + +## Background + +### int main(int argc, char *argv[]) + +Programs written in C for Linux define a function called main that is the entry point of the program. Documents such as the N2310 draft of the C language standard describe the main function. Page 11, section 5.1.2.2.1, "Program startup", provides a common definition of main: + +``` +int main(int argc, char *argv[]) { /* ... */ } +``` + +=> http://www.open-std.org/jtc1/sc22/wg14/www/docs/n2310.pdf The N2310 draft of the C language standard + +The argc parameter contains the **c**ount of the arguments provided to the program. The argv parameter contains their **v**alues. + +Create a file named argv.c with the following contents: + +``` +#include <stdio.h> + +int main(int argc, char *argv[]) { + for(int i=0; i<argc; i++) { + printf("Argument %d -%s-\n", i, argv[i]); + } +} +``` + +Compile the file running the following command: + +``` +$ cc argv.c +``` + +This produces an executable file named "a.out". This executable will print the arguments you provide via the command line: + +``` +$ ./a.out +Argument 0 -./a.out- +``` + +``` +$ ./a.out arg1 arg2 arg3 +Argument 0 -./a.out- +Argument 1 -arg1- +Argument 2 -arg2- +Argument 3 -arg3- +``` + +Note that the first argument is the name of the executable file itself. + +Note that when using quoting, the program prints things like: + +``` +$ ./a.out "a b" c +Argument 0 -./a.out- +Argument 1 -a b- +Argument 2 -c- +``` + +So the first argument is "a b" (without quotes). + +### exec(3) + +UNIX-like operating systems provide the "exec" family of functions to invoke commands. "man 3 exec" describes the exec family of functions in Linux. Linux provides the execl, execlp, execle, execv, execvp, and execvpe functions. These functions allow us to execute a command from within a C program. + +Create a file named execlp.c with the following contents: + +``` +#include <stdlib.h> +#include <unistd.h> + +int main() { + exit(execlp("cat", "cat", "/etc/passwd", NULL)); +} +``` + +Compile the file running the following command: + +``` +$ cc execlp.c +``` + +This produces an executable file named "a.out". +Execute it: + +``` +$ ./a.out +``` + +This is equivalent to running in a shell the statement "cat /etc/passwd". + +This article does not describe the intricacies of the exec family of functions. However, let's analyze the call to execlp. + +The exec functions whose name contains a "p" look up the command to execute by searching for executables named like the first argument in the directories listed in the PATH environment variable. In the example, execlp looks up the cat executable in directories such as /usr/bin. + +The second argument is also the name of the program. + +Note that in the preceding argv.c example, the zeroth argument is the name of the program being executed. Some executables in Linux systems are present under different names (using symbolic links). For example, xzcat is a symbolic link to xz. Running xzcat or xz runs the same executable file, but the executable uses the zeroth argument to change its behavior. + +This technique is a simple way to "share" code between similar programs. The BusyBox project provides many common utilities, such as ls and cat, in a single executable. By sharing code among all utilities, the BusyBox executable is smaller. + +The rest of the parameters to execlp are the arguments for the executable file. + +In a way, exec functions "call" the main function of other programs. The parameters to exec are "passed" to the main function. + +### Shells + +Programs such as bash provide a way to execute other programs. When you type a statement such as "cat /etc/passwd", bash parses the statement into a command to execute and arguments. Then, bash uses an exec function to run the program with arguments. + +The simplest bash statements are words separated by spaces, of the form "arg0 arg1 arg2 ... argn". + +On such a statement, bash executes something like: + +``` +execlp(arg0, arg0, arg1, _..._, argn, NULL) +``` + +And the program will receive the string arg0 as the zeroth argument, arg1 as the first argument, and so forth. + +However, using cat to view the contents of files, the user might want to view a file whose name contains spaces. + +The statement "cat a b" has two arguments: a and b. For each argument, cat prints the file of that name. So the "cat a b" statement prints the contents of the a and b files, not of a file named "a b". + +## Further reading + +=> http://teaching.idallen.com/cst8177/13w/notes/000_find_and_xargs.html Using find -exec or xargs to process pathnames with other commands +=> https://infosec.exchange/@david_chisnall/115116683569142801 Early UNIX did glob expansion in the shell not because that’s more sensible than providing a glob and option parsing API in the standard library, but because they didn’t have enough disk space or RAM to duplicate code and they didn’t have shared libraries... For example, on FreeBSD, I often do pkg info foo* to print info about packages that start with some string. If I forget to quote the last argument, this behaves differently depending on whether the current directory contains one or more files that have the prefix that I used. If they do, the shell expands them and pkg info returns nothing because I don’t have any installed packages that match those files. If they don’t, the shell passes the star to the program, which does glob expansion but against a namespace that is not the filesystem namespace. The pkg tool knows that this argument is a set of names of installed packages, not files in the current directory, but it can’t communicate that to the shell and so the shell does the wrong thing. Similarly, on DOS the rename command took a load of source files and a destination file or pattern. You could do rename *.c *.txt and it would expand the first pattern, then do the replacement based on the two patterns. UNIX’s mv can’t do that and I deleted a bunch of files by accident when I started using Linux because it’s not obvious to a user what actually happens when you write mv *.c *.txt. There is a GNU (I think?) rename command and its syntax is far more baroque than the DOS one because it is fighting against the shell doing expansion without any knowledge of the argument structure. + +## TODO + +=> https://news.ycombinator.com/item?id=36722570 SSH particularities diff --git a/programming/so_you_want_to_play_with_functional_programming.md b/blog/content/notes/tech/so-you-want-to-play-with-functional-programming.gmi index ecf2cce3..2e8abec5 100644 --- a/programming/so_you_want_to_play_with_functional_programming.md +++ b/blog/content/notes/tech/so-you-want-to-play-with-functional-programming.gmi @@ -1,13 +1,10 @@ # So you want to play with functional programming -If you are a programmer working on popular languages such as Python or Java, you are likely to have read articles about "functional programming". -These articles can give you the idea that learning functional programming improves your skills as a programmer. -I share this opinion. +If you are a programmer working on popular languages such as Python or Java, you are likely to have read articles about "functional programming". These articles can give you the idea that learning functional programming improves your skills as a programmer. I share this opinion. This article tries to help people who have read about functional programming figure out how to proceed. -Note that this article expresses personal opinion. -Particularly, I am not an expert in this topic: +Note that this article expresses personal opinion. Particularly, I am not an expert in this topic: * I have programmed some Haskell (about 50 Project Euler problems, plus experimentation on and off during the years). * I have studies of SML and functional programming. @@ -17,26 +14,18 @@ Particularly, I am not an expert in this topic: Shortly after writing this, I was shown: -https://technomancy.us/194 +=> https://technomancy.us/194 In which there is no such thing as a functional programming language -I agree with most of that the article explains. -I might extend this article with some similar ideas, but for the moment, I recommend reading that carefully before reading the rest of this article. +I agree with most of that the article explains. I might extend this article with some similar ideas, but for the moment, I recommend reading that carefully before reading the rest of this article. ## The basics of functional programming -[The Wikipedia article on functional programming](https://en.wikipedia.org/wiki/Functional_programming) is a great place to get started. +=> https://en.wikipedia.org/wiki/Functional_programming The Wikipedia article on functional programming is a great place to get started. -The article describes a few concepts related to functional programming. -I consider the following two the pillars of functional programming: +The article describes a few concepts related to functional programming. I consider the following two the pillars of functional programming: -* First-class and higher order functions. - In languages with first-class functions, functions are values that you can use like other types such as integers. - Higher-order functions are functions that take functions as arguments or return functions. - -* Pure functions. - Pure functions always return the same value for a given set of arguments. - Pure functions also have no side effects; they do not modify anything in the system they run. - For example, a function that creates a file is not pure. +* First-class and higher order functions. In languages with first-class functions, functions are values that you can use like other types such as integers. Higher-order functions are functions that take functions as arguments or return functions. +* Pure functions. Pure functions always return the same value for a given set of arguments. Pure functions also have no side effects; they do not modify anything in the system they run. For example, a function that creates a file is not pure. These concepts can be applied in most popular programming languages. @@ -51,8 +40,7 @@ def twice(f): f() ``` -`twice` is a higher order function because it takes a function as an argument. -Functions are first-class functions because you can use `hello` as a value: +twice is a higher order function because it takes a function as an argument. Functions are first-class functions because you can use hello as a value: ``` >>> twice(hello) @@ -62,36 +50,24 @@ hello Similarly, you can write pure functions in almost any language. -When you have first-class functions, you can define some higher-order functions that generalize some common code. -Three very common higher-order functions are: +When you have first-class functions, you can define some higher-order functions that generalize some common code. Three very common higher-order functions are: -* Filter. - Filter applies a function to each element of a list, and returns a list composed of the elements for which the function returned true. -* Map. - Map applies a function to each element of a list, and returns a list of the result of the application of the function to each element. -* Fold. - A fold starts from an initial value, then calls a function with the initial value and the first element of the list. - Then it calls the function with the result of the previous call, and the next element of the list. - This continues until the list end, returning the last result of the function. +* Filter applies a function to each element of a list, and returns a list composed of the elements for which the function returned true. +* Map applies a function to each element of a list, and returns a list of the result of the application of the function to each element. +* Fold. A fold starts from an initial value, then calls a function with the initial value and the first element of the list. Then it calls the function with the result of the previous call, and the next element of the list. This continues until the list end, returning the last result of the function. (For example, folding with the sum operator and an initial value of 0, sums the elements of a list.) -Note that you can implement many list manipulations by composing filters, maps, and folds with different functions. -(And by adding more higher-order functions, you can implement more list manipulations.) +Note that you can implement many list manipulations by composing filters, maps, and folds with different functions. (And by adding more higher-order functions, you can implement more list manipulations.) Also, you can manipulate other data structures with equivalent or other higher-order functions. Implementing code using higher-order functions and pure functions already has some interesting benefits. -* Impure functions frequently require more mental overhead to understand, because you need to understand state. - With pure functions, you do not have to think about state. - -* To understand a program written as a composition of functions, you can start by understanding individual functions and then understand how they fit together. - The same program written as a sequence of statements is often more difficult to understand. - (However, sometimes the opposite effect occurs.) +* Impure functions frequently require more mental overhead to understand, because you need to understand state. With pure functions, you do not have to think about state. +* To understand a program written as a composition of functions, you can start by understanding individual functions and then understand how they fit together. The same program written as a sequence of statements is often more difficult to understand. (However, sometimes the opposite effect occurs.) -You can use these concepts in most popular programming languages. -(Most popular languages also provide higher-order functions such as filters, maps, and folds.) +You can use these concepts in most popular programming languages. (Most popular languages also provide higher-order functions such as filters, maps, and folds.) So you can get started with functional programming by using the programming languages you already know: @@ -107,8 +83,7 @@ Writing code using these concepts often leads to: * Writing cumbersome code if the programming language you use lacks certain features. * Unlocking additional functional programming techniques. -Therefore, many programming languages provide features that make functional programming more straightforward, or features enabled by functional programming. -Languages providing features related to functional programming are commonly named "functional programming languages". +Therefore, many programming languages provide features that make functional programming more straightforward, or features enabled by functional programming. Languages providing features related to functional programming are commonly named "functional programming languages". Although you can use functional programming with non-functional programming languages, this can often lead to: @@ -117,23 +92,18 @@ Although you can use functional programming with non-functional programming lang ### The need for powerful type systems and type inference -Higher-order functions often have complex type requirements. -For example, to filter a list of a given type, you must pass a function that takes a single argument of that type and returns a boolean. -If the arguments do not have the correct types, then the code does not work correctly. +Higher-order functions often have complex type requirements. For example, to filter a list of a given type, you must pass a function that takes a single argument of that type and returns a boolean. If the arguments do not have the correct types, then the code does not work correctly. -In languages with dynamic types, the program fails at runtime. -In languages with static types, you frequently must specify the types, and higher-order functions often require complex types involving different function types. +In languages with dynamic types, the program fails at runtime. In languages with static types, you frequently must specify the types, and higher-order functions often require complex types involving different function types. Functional programming languages frequently: * Have static types, to prevent frequent runtime failures. -* Automatically infer types instead of requiring programmers to declare them. - (However, automatic type inference can cause issues in some scenarios, so frequently programming languages allow writing explicit types, or even require explicit types in some cases.) +* Automatically infer types instead of requiring programmers to declare them. (However, automatic type inference can cause issues in some scenarios, so frequently programming languages allow writing explicit types, or even require explicit types in some cases.) Because functional programs often use more complex types, functional programming languages often have more powerful type systems than non-functional programming languages. -Derived from those properties, functional programming languages result in the "if it compiles, it works *correctly*" phenomenon. -This phenomenon helps avoid incorrect programs. +Derived from those properties, functional programming languages result in the "if it compiles, it works *correctly*" phenomenon. This phenomenon helps avoid incorrect programs. ## Functional programming languages @@ -141,34 +111,26 @@ This phenomenon helps avoid incorrect programs. Functional programming practitioners often recommend Haskell as a functional programming language. -[According to the Wikipedia](https://en.wikipedia.org/wiki/Haskell), "Haskell is a general-purpose, statically-typed, purely functional programming language with type inference and lazy evaluation". -Also, Haskell was designed by a committee whose purpose was "to consolidate existing functional languages into a common one to serve as a basis for future research in functional-language design". +According to the Wikipedia, "Haskell is a general-purpose, statically-typed, purely functional programming language with type inference and lazy evaluation". Also, Haskell was designed by a committee whose purpose was "to consolidate existing functional languages into a common one to serve as a basis for future research in functional-language design". -* Haskell is perhaps the language with more built-in functional programming features. -As mentioned, Haskell is used for research about functional programming, therefore many new concepts appear in Haskell first. +* Haskell is perhaps the language with more built-in functional programming features. As mentioned, Haskell is used for research about functional programming, therefore many new concepts appear in Haskell first. * Haskell is also very strict about functional programming, so Haskell drives programmers more strongly towards avoiding non-functional programming. * Haskell syntax is designed so Haskell programs can be extremely terse and contain almost no extraneous syntax. -* Haskell is a very popular language, with a very large ecosystem. - You can take advantage of many existing libraries and tools for developing real-world programs faster. +* Haskell is a very popular language, with a very large ecosystem. You can take advantage of many existing libraries and tools for developing real-world programs faster. However, Haskell's benefits frequently also are negative for learning. -* Haskell uses "lazy" evaluation, where most programming languages use "eager" evaluation. - Haskell does not evaluate expressions until needed (and might not evaluate some expressions). - Lazy evaluation can lead to efficiency. - However, lazy evaluation can cause unexpected performance problems. - [Foldr Foldl Foldl'](https://wiki.haskell.org/Foldr_Foldl_Foldl%27) explains how choosing incorrectly among different implementations of fold can lead to impactful performance problems. - When writing Haskell code for learning, you can likely stumble into issues not present in languages that use eager evaluation. +* Haskell uses "lazy" evaluation, where most programming languages use "eager" evaluation. Haskell does not evaluate expressions until needed (and might not evaluate some expressions). Lazy evaluation can lead to efficiency and clearer programs. However, lazy evaluation can cause unexpected performance problems. + +=> https://wiki.haskell.org/Foldr_Foldl_Foldl%27 "Foldr Foldl Foldl'" explains how choosing incorrectly among different implementations of fold can lead to impactful performance problems. + +When writing Haskell code for learning, you can likely stumble into issues not present in languages that use eager evaluation. -* Haskell is very strict about purity. - To implement programs that have side effects, such as accessing files, you must use specific language features. - Many articles try to explain those features, because many people have trouble understanding them. +* Haskell is very strict about purity. To implement programs that have side effects, such as accessing files, you must use specific language features. Many articles try to explain those features, because many people have trouble understanding them. -* Many libraries and tools in the ecosystem take advantage of powerful features enabled by Haskell. - However, this might cause that using these libraries and tools require the understanding of the features they are based upon. +* Many libraries and tools in the ecosystem take advantage of powerful features enabled by Haskell. However, this might cause that using these libraries and tools require the understanding of the features they are based upon. -Also, Haskell syntax is very terse, which leads to Haskell compilers not providing clear error messages. -For example: +Also, Haskell syntax is very terse, which leads to Haskell compilers not providing clear error messages. For example: ``` $ ghci @@ -186,38 +148,29 @@ $ ghci In complex programs, programmers new to Haskell might have trouble identifying that a function has been called with an extra argument from that error message. -Personally, Haskell is my favorite functional programming language. -However, I learned Haskell after learning (with teachers and support from others) other functional programming languages. -I think that Haskell is ideal to learn the most powerful concepts in functional programming, but it is not as ideal as a first functional programming language. +Personally, Haskell is my favorite functional programming language. However, I learned Haskell after learning (with teachers and support from others) other functional programming languages. I think that Haskell is ideal to learn the most powerful concepts in functional programming, but it is not as ideal as a first functional programming language. (Note that these recommendations come from someone who only has implemented about 50 Project Euler problems in Haskell, and has experimented on and off with the language, but not been paid for it.) ### Lisp -Many programmers like Lisp and languages in the Lisp family, such as Scheme or Clojure. -Lisp programmers often recommend Lisp to learn functional programming. +Many programmers like Lisp and languages in the Lisp family, such as Scheme or Clojure. Lisp programmers often recommend Lisp to learn functional programming. -Lisp is a very minimalistic, yet infinitely flexible language. -Lisp is extensible, so you can add most programming language features to Lisp, including functional programming features. +Lisp is a very minimalistic, yet infinitely flexible language. Lisp is extensible, so you can add most programming language features to Lisp, including functional programming features. Therefore, you can do functional programming in Lisp, and also benefit from all other Lisp features. However, languages in the Lisp family tend to not have static typing and associated features, thus do not frequently exhibit the "if it compiles, it works *correctly*" phenomenon. -Lisp has one of the simplest syntaxes of any programming language. -The simple syntax of Lisp is directly tied to its power. -Many favor the Lisp syntax and argue that the syntax makes Lisp better for learning programming. -Personally, I find the Lisp syntax hard to read and write, and likely an additional difficulty on top of learning functional programming. +Lisp has one of the simplest syntaxes of any programming language. The simple syntax of Lisp is directly tied to its power. Many favor the Lisp syntax and argue that the syntax makes Lisp better for learning programming. Personally, I find the Lisp syntax hard to read and write, and likely an additional difficulty on top of learning functional programming. -I recommend learning Lisp because it is a unique programming language that can teach you many programming language concepts that are not present in many other languages. -However, I do not recommend Lisp for learning functional programming (unless you already know Lisp). +I recommend learning Lisp because it is a unique programming language that can teach you many programming language concepts that are not present in many other languages. However, I do not recommend Lisp for learning functional programming (unless you already know Lisp). (Note that these recommendations come from someone who has some formal training on Lisp but only uses Lisp infrequently [as a recent Emacs user].) ### The ML family of programming languages -ML is a language that appeared in 1973. -Since then, three dialects have become the most popular implementations of ML: +ML is a language that appeared in 1973. Since then, three dialects have become the most popular implementations of ML: * OCaml * Standard ML @@ -243,7 +196,6 @@ Error: This function has type int -> int -> int It is applied to too many arguments; maybe you forgot a `;'. ``` -In my opinion, OCaml and F# are better languages for the initial learning of functional programming than Haskell. -After learning an ML, you are likely more prepared to learn Haskell and more sophisticated functional programming. +In my opinion, OCaml and F# are better languages for the initial learning of functional programming than Haskell. After learning an ML, you are likely more prepared to learn Haskell and more sophisticated functional programming. (Note that those recommendations come from someone who only has experimented with OCaml and F#, and learned SML formally.) diff --git a/blog/content/notes/tech/ssh-for-beginners.gmi b/blog/content/notes/tech/ssh-for-beginners.gmi new file mode 100644 index 00000000..0c74b2e4 --- /dev/null +++ b/blog/content/notes/tech/ssh-for-beginners.gmi @@ -0,0 +1,88 @@ +# SSH for beginners + +Some simple advice for people who are starting to use ssh. + +## Use the config + +If you create a file "~/.ssh/config", with contents like: + +``` +Host xxx + HostName yyy + Port 1234 + User zzz +``` + +, then if you type "ssh xxx", the result will be like executing "ssh -p 1234 zzz@yyy". + +Any ssh command line arguments can be encoded in an SSH client configuration file, so you can access any server by just passing a host to ssh without any additional parameters. + +Additionally, most modern systems configure SSH tab completion, so if you type "ssh <tab><tab>", your shell will complete with the hosts in your configuration file. + +## Use public key authentication + +By default, ssh uses passwords for authentication. If you use a good password, then password authentication is a decent authentication method. + +However, you can use other methods, such as public key authentication. With public key authentication, you have a public and private key. + +If you are working on system A with your *private* key, and you copy your *public* key to system B, then you can ssh from system A to system B without entering a password. + +### Security + +Note that if someone obtains your private key, they will be able to log in to systems that trust your key. Knowledge of your private key is similar to knowledge of a password. Take care making your private key truly private. + +If you suspect someone else has been able to obtain your private key, then generate a new key and remove the leaked public key from all systems. + +Note that you can generate as many keys as you want. Managing multiple keys requires more effort, but in some cases it might be more convenient. For example, if a key is suspected to be leaked, then you might only need to revoke a key and continue using other keys. + +### Generating SSH keys + +To generate your private and public keys: + +``` +$ ssh-keygen +Generating public/private rsa key pair. +Enter file in which to save the key (/home/alex/.ssh/id_rsa): +Created directory '/home/alex/.ssh'. +Enter passphrase (empty for no passphrase): +Enter same passphrase again: +Your identification has been saved in /home/alex/.ssh/id_rsa +Your public key has been saved in /home/alex/.ssh/id_rsa.pub +The key fingerprint is: +SHA256:... +The key's randomart image is: ++---[RSA 3072]----+ +... +``` + +### Key type choice + +OpenSSH, the standard ssh client, changed its default type of key generation to Ed25519 in version 9.5 released in late 2023. Previously, ssh-keygen generated RSA keys, as in the example above. Many Linux distributions still use OpenSSH versions earlier than 9.5. + +You can find advisories like: + +> It is quite possible the RSA algorithm will become practically breakable in the foreseeable future. All SSH clients support this algorithm. + +=> https://www.ssh.com/academy/ssh/keygen + +Although as of the time of writing this, RSA is considered safe. However, you can consider generating an Ed25519 key instead, following the most recent OpenSSH defaults. + +### Passphrases + +By default, if you provide an empty passphrase to ssh-keygen, your private key will be stored unprotected. Anyone that can read the private key file can obtain your key. + +You can use a passphrase to protect your key. If someone obtains a private key file but they don't know the passphrase, then they cannot use the key. + +Using a passphrase means that you need to type the passphrase every time you use the key, or use a system such as ssh-agent. This creates a tradeoff between security and convenience. + +(Note that a popular criticism of SSH public key authentication is that it is not easy for systems administrators to enforce the use of SSH passphrases.) + +## Further SSH features + +Many developers have added many useful features to SSH during many years, such as: + +* The scp command to transfer files using SSH +* Tunnels to establish bidirectional communication between systems without such connectivity. (For example, to connect to your workstation from a remote system.) +* Jump hosts that expedite the connection to a system that is not directly accessible, by using SSH to establish connection through intermediate systems. + +Also, SSH integrates very well with UNIX pipes and tools such as rsync, Git, and many others. diff --git a/blog/content/notes/tech/take-the-less-traveled-road.gmi b/blog/content/notes/tech/take-the-less-traveled-road.gmi new file mode 100644 index 00000000..e9107470 --- /dev/null +++ b/blog/content/notes/tech/take-the-less-traveled-road.gmi @@ -0,0 +1,29 @@ +# Take the less traveled road + +> Two roads diverged in a wood, and I— + +> I took the one less traveled by, + +> And that has made all the difference. + +=> https://www.gutenberg.org/cache/epub/59824/pg59824-images.html#THE_ROAD_NOT_TAKEN Robert Frost, The Road Not Taken + +=> https://xkcd.com/743/ Infrastructures, by Randall Munroe + +The prisoner's dilemma describes situations where people can choose to act in their own interest or collaborate with others. I believe the prisoner's dilemma is present everywhere in our daily life and can explain many behaviors and situations. + +One such scenario is when we choose services to use on the Internet. + +Frequently, we choose services that are easy, popular, and even have no cost. + +In the past, even non-technical people got shared hosting and ran WordPress to blog, with nearly total control about their communication. Nowadays, mostly everyone uses something like Twitter and cedes control to a company. + +Using Twitter is easier than self-hosting WordPress. Twitter is so popular, that your message will likely reach more users on Twitter than on an independent blog. And you can use Twitter for free, whereas shared hosting costs you money and time. + +Twitter became dominant and nowadays, many are frustrated by how Twitter has changed. + +Dominance implies lack of competition, which is nearly always bad for the consumer. + +Which means choosing which services to use frequently is a difficult choice. + +We all have limited time and energy, but I propose that whenever you can, you should take the less traveled road. The easy alternative likely benefits you in the short term, but can easily contribute to a monoculture that will damage everyone in the end, including you. diff --git a/programming/the_tragedy_of_the_geeks.md b/blog/content/notes/tech/the-tragedy-of-the-geeks.gmi index c1193d5f..e5ce1b10 100644 --- a/programming/the_tragedy_of_the_geeks.md +++ b/blog/content/notes/tech/the-tragedy-of-the-geeks.gmi @@ -1,12 +1,10 @@ # The tragedy of the geeks -Since the first computer entered our home, I was hooked. -This happened more than four decades ago, and continuously tinkering with computers has given me a well-paid and comfortable job. +Since the first computer entered our home, I was hooked. This happened more than four decades ago, and continuously tinkering with computers has given me a well-paid and comfortable job. However, getting such jobs seems linked to spending a significant amount of your personal time practicing your skills. -Many people seek careers related to computing because jobs have attractive conditions. -However, they might later regret the time and energies spent trying to get into the field when they learn that getting a good job requires unexpected effort. +Many people seek careers related to computing because jobs have attractive conditions. However, they might later regret the time and energies spent trying to get into the field when they learn that getting a good job requires unexpected effort. This document tries to explain to people who want to work with computers this phenomenon, to help them make a better decision. @@ -18,18 +16,15 @@ Working with computers is the only career I can think of where all of the follow * There is a reputation of abundant well-paid job offers with good conditions. * Working on personal projects sounds fun. -This means that many of us end up spending a significant amount of time working on personal projects. -This time investment increases our skills and the things we know. +This means that many of us end up spending a significant amount of time working on personal projects. This time investment increases our skills and the things we know. ## Hiring Hiring is one of the most highly debated topics in this industry. -Many people believe that many candidates cannot do the job. -There are many stories about new hires who cannot write simple programs. +Many people believe that many candidates cannot do the job. There are many stories about new hires who cannot write simple programs. -Whether this is common or not is not as important as whether people making decisions believe there are large differences between candidates. -When people who hire think that their hiring decision is going to have a large effect on them, then they want to make sure that they pick the right person. +Whether this is common or not is not as important as whether people making decisions believe there are large differences between candidates. When people who hire think that their hiring decision is going to have a large effect on them, then they want to make sure that they pick the right person. My perception is that most of the organizations that offer good job conditions (and many who do not) try to be very selective in hiring people. @@ -37,8 +32,7 @@ My perception is that most of the organizations that offer good job conditions ( When you are hiring people, candidates who have spent significant time on personal projects tend to stand out over candidates who have not. -This improved perception during the hiring process does not necessarily relate to improved performance on the job. -However, I believe that people who tinker on their spare time tend to land better jobs. +This improved perception during the hiring process does not necessarily relate to improved performance on the job. However, I believe that people who tinker on their spare time tend to land better jobs. ## Handing out advice @@ -46,20 +40,17 @@ Because there are good jobs working with computers, many people think about maki There are many curriculums and formal education programs, from shorter (typically one year) to longer (four or five years). -Some of them provide advice to land a good job, and students who follow programs who do not, tend to ask for advice. -In any case, one of the most frequent pieces of advice on the topic, is tinkering on your own time. +Some of them provide advice to land a good job, and students who follow programs who do not, tend to ask for advice. In any case, one of the most frequent pieces of advice on the topic, is tinkering on your own time. I believe this is actually good advice, as in that it's more likely to be an efficient way to increase your prospects. -However, remember that hiring is roughly a competitive process. -An organization evaluates a group of candidates, and tries to pick the best one. +However, remember that hiring is roughly a competitive process. An organization evaluates a group of candidates, and tries to pick the best one. So to stand out, if more candidates tinker (because this is effective advice), the more you need to tinker to stand out. I cannot estimate how much you need to tinker on your own time to land a good job, but my guess is that it is more than what someone wanting to get into the field expects. -As long as this dynamic continues, the tinkering required to land a good job will increase. -Only reduced competition can reduce the tinkering required, and reduced competition can happen by few factors, such as increased demand for workers, or a reduction in job seekers. +As long as this dynamic continues, the tinkering required to land a good job will increase. Only reduced competition can reduce the tinkering required, and reduced competition can happen by few factors, such as increased demand for workers, or a reduction in job seekers. ## Breaking the cycle @@ -67,9 +58,8 @@ I cannot think of much that we can individually do to break the cycle. Maybe if people coming into the field are aware of this phenomenon, they will be able to make a better decision about what to do. -If a sufficient amount of people decide that the time investment is not worthwhile, then perhaps the competition will decrease. -And if people are well informed and decide to move forward, at least they will be less likely to become frustrated or regret their decision. +If a sufficient amount of people decide that the time investment is not worthwhile, then perhaps the competition will decrease. And if people are well informed and decide to move forward, at least they will be less likely to become frustrated or regret their decision. ## Further reading -* [A paean to programming](https://bertrandmeyer.com/2025/04/23/a-paean-to-programming/), by [Bertrand Meyer](https://en.wikipedia.org/wiki/Bertrand_Meyer) +=> https://bertrandmeyer.com/2025/04/23/a-paean-to-programming/ A paean to programming, by Bertrand Meyer diff --git a/emacs/emacs.el b/emacs/emacs.el index 50639786..807d1ed2 100644 --- a/emacs/emacs.el +++ b/emacs/emacs.el @@ -67,9 +67,6 @@ :config (global-kkp-mode +1)) -;; Helps eglot locate remote language servers, such as ones in ~/.local/bin -(add-to-list 'tramp-remote-path 'tramp-own-remote-path) - ;; Install xclip so cutting/copying in Emacs on a terminal affects the graphical clipboard (use-package xclip :ensure t diff --git a/linux/akai-mpk-mini.md b/linux/akai-mpk-mini.md deleted file mode 100644 index 6c680167..00000000 --- a/linux/akai-mpk-mini.md +++ /dev/null @@ -1,3 +0,0 @@ -# AKAI MPK Mini - -* https://www.dpin.de/nf/linux-music-making-sonic-pi-lmms-akai-mpk-usb-midi/ has software reocommendations and instructions to adjust velocity. diff --git a/linux/misc.md b/linux/misc.md deleted file mode 100644 index 326a5ad5..00000000 --- a/linux/misc.md +++ /dev/null @@ -1,77 +0,0 @@ -# Misc - -## Reverse sshfs - -https://blog.dhampir.no/content/reverse-sshfs-mounts-fs-push - -You need the SFTP server program on your local machine (on Debian, the `openssh-server` package) and sshfs on the remote machine. - -## Find non-Debian packages - -``` -aptitude search '~S ~i !~ODebian' -``` - -## Memory usage queries - -### systemd - -``` -systemd-cgtop -m -``` - -Drill down with: - -``` -systemd-cgtop -m user.slice/user-1000.slice -``` - -### smem - -``` -sudo smem -P beam.smp -kta -``` - -## Quick rerouting of browser traffic through another host - -`ssh -D 1234 host` creates a Socks proxy on `localhost:1234` that sends traffic through `host`. - -By enabling "allow extension to control proxy settings" in the [multi account containers](https://addons.mozilla.org/en-US/firefox/addon/multi-account-containers/) Firefox add-on, you can make containers use specific proxies. - -## Email forwarding via IMAP - -When you have multiple email addresses, you have multiple options to use them. - -Traditionally, people used redirection/forwarding to make email arriving at `from@example.com` go to `to@example.net` instead. - -If mail from `example.com` and `example.net` is handled by different servers, typically you can configure the `example.com` mail server to resend any message arriving to the `from` address to `to@example.net`. - -However, nowadays with spam filtering, the `example.net` mail server can reject these emails as spam, sometimes silently. - -For safer redirects, you can: - -* Use the same mail server for both accounts. - However, this sometimes cannot be done or has extra cost and complexity. - -* Configure the destination email server to fetch email from the origin mail server. - For example, Gmail can do this, but the fetching period can be as long as 15 minutes. - This can be inconvenient when receiving confirmation emails, etc. - - Additionally, operators of the destination email server now have your credentials. - -A third option is to run this fetching process yourself. - -<https://github.com/lefcha/imapfilter> supports connecting to an IMAP account, waiting until messages to arrive, and moving them to another IMAP account. - -Benefits: - -* IMAPFilter can use IMAP idle to request the IMAP server to notify when messages arriving, so forwarding happens without a polling delay. -* Because IMAP is used on both sides, no spam filtering happens. -* IMAPFilter is [packaged for many distributions](https://repology.org/project/imapfilter/versions). - -Drawbacks: - -* Requires additional infrastructure. -* If IMAPFilter stops working, email stops being forwarded without warning. - -Refer to [this Python module](../scripts/p7s/mail/__init__.py) for scripts that configure IMAPFilter as a systemd service, with credentials from Bitwarden. diff --git a/linux/pedal.md b/linux/pedal.md deleted file mode 100644 index acd0ceb8..00000000 --- a/linux/pedal.md +++ /dev/null @@ -1,13 +0,0 @@ -# Pedal - -With a PCSensor pedal and <https://github.com/rgerganov/footswitch>: - -``` -sudo footswitch -1 -m ctrl -2 -S '7F' -3 -m win -k m -``` - -Maps: - -* Left pedal to be a ctrl key -* Center pedal to the toggle mute key -* Right pedal to be Win+M (you can map this in Gnome to toggle mic status) diff --git a/linux/remote-networking.md b/linux/remote-networking.md deleted file mode 100644 index dac35083..00000000 --- a/linux/remote-networking.md +++ /dev/null @@ -1,24 +0,0 @@ -# Remote networking - -If you can create a pipe between two hosts (using SSH, for example), you can use VDE (Virtual Distributed Ethernet) to connect the two hosts over a virtual network. - -You need the following programs on both hosts: - -* `dpipe` and `vde_plug` (on Debian, use the `vdeplug` package) -* `vde-switch` (on Debian, use the `vde-switch` package) - -Run `vde_switch -t tap0` as root on both hosts. -This command creates a virtual switch connected to `tap0`. - -Use the `dpipe` command to connect two instances of the `vde_plug` command running as root on both hosts. - -```console -$ dpipe sudo vde_plug = ssh root@remote vde_plug -``` - -Then bring the `tap0` interface up and configure IP addresses on both hosts. - -```console -# ip link set tap0 up -# ip addr add $IP/$NETMASK dev tap0 -``` diff --git a/linux/running_commands_in_linux.adoc b/linux/running_commands_in_linux.adoc deleted file mode 100644 index b31c39bb..00000000 --- a/linux/running_commands_in_linux.adoc +++ /dev/null @@ -1,292 +0,0 @@ -= Notes on Running Commands in Linux - -== Motivating Examples - -=== CWE-78: Improper Neutralization of Special Elements used in an OS Command ('OS Command Injection') - -The https://cwe.mitre.org/data/definitions/1337.html[2021 CWE Top 25 Most Dangerous Software Weaknesses] helps focus on the biggest security issues that developers face. -Number 5 on that list is https://cwe.mitre.org/data/definitions/78.html[Improper Neutralization of Special Elements used in an OS Command ('OS Command Injection')]. - -Software developers often write code that invokes other programs. -For example, shell scripts tend to be mostly composed of invocations of programs such as `find`, `grep`, etc. -Even software developed in languages such as Python, C, or Java often invokes other programs. - -Python software developers use the `subprocess` module to perform this task. -Other languages provide similar facilities. - -Consider the two following Python sessions to execute an equivalent to the `bash` statement `cat /etc/passwd`: - ----- -$ python3 ->>> import subprocess ->>> subprocess.run(["cat", "/etc/passwd"]) ----- - ----- -$ python3 ->>> import subprocess ->>> subprocess.run("cat /etc/passwd", shell=True) ----- - -Both scripts use the same `run` function, with different values of the `shell` parameter (the `shell` parameter defaults to `True`). -When executing a command with many arguments, `shell=True` seems to be terser. -`a b c d e` is shorter and easier to read than `["a", "b", "c", "d", "e"]`. -Readable code is easier to maintain, so a software developer could prefer the `shell=True` version. - -However, using `shell=True` can introduce the "OS Command Injection" weakness easily. - -Create a file named "injection.py" with the following contents: - ----- -import sys -import subprocess - -subprocess.run(f"cat {sys.argv[1]}", shell=True) ----- - -This program uses the `cat` command to display the contents of a file. -For example, if you run (using Python 3.6 or higher): - ----- -$ python3 injection.py /etc/passwd ----- - -The terminal shows the contents of the `/etc/passwd` file. - -However, if you run: - ----- -$ python3 injection.py '/etc/passwd ; touch injected' ----- - -The terminal shows the same file, but a file named `injected` also appears in the current directory. - -Create a file named "safe.py" with the following contents: - ----- -import sys -import subprocess - -subprocess.run(["cat", sys.argv[1]]) ----- - -Running `python3 safe.py /etc/passwd` has the same behavior as using `injection.py`. -However, repeating the command that creates a file using `safe.py` results in: - ----- -$ python3 safe.py '/etc/passwd ; touch injected' -cat: '/etc/passwd ; touch injected': No such file or directory ----- - -`injection.py` is vulnerable to "OS Command Injection" because it uses `shell=True`, whereas `safe.py` is not. - -If a malicious user can get strings such as `/etc/passwd ; touch injected` to code that uses `shell=True`, then the user can execute arbitrary code in the system. -Code that does not handle user input might not be exposed to such issues, but user input might creep in and introduce unexpected vulnerabilities. -Avoiding the use of `shell=True` and similar features can be safer than making sure that user input is correctly handled in all cases. - -=== Writing Shell Scripts that Handle Files with Spaces in Their Names - -Create a file called `backup.sh` with the following contents: - ----- -#!/bin/bash - -for a in $1/* ; do - cp $a $a.bak -done ----- - -Run the following statements in the terminal to create a sample directory with files. - ----- -$ mkdir backup_example_1 -$ for a in $(seq 1 9) ; do echo $a >backup_example_1/$a ; done ----- - -These statements create the `backup_example_1` directory, and files named `1`, ..., `9`. - -The `backup.sh` script creates a copy of each file in a directory. -If you run: - ----- -$ bash backup.sh backup_example_1/ ----- - -Then the script will copy `1` to `1.bak`, and so on. - -However, if you create a new directory with files whose names have spaces: - ----- -$ mkdir backup_example_2 -$ for a in $(seq 1 9) ; do echo $a >backup_example_1/"file $a" ; done ----- - -Then the `backup.sh` script does not work correctly: - ----- -$ bash backup.sh backup_example_2/ -cp: cannot stat 'backup_example_2//*': No such file or directory ----- - -In order to fix the script, change the contents of `backup.sh` to: - ----- -#!/bin/bash - -for a in "$1/*" ; do - cp "$a" "$a.bak" -done ----- - - -== Background - -=== `int main(int argc, char *argv[])` - -Programs written in C for Linux define a function called `main` that is the entry point of the program. -Documents such as http://www.open-std.org/jtc1/sc22/wg14/www/docs/n2310.pdf[the _N2310_ draft of the C language standard] describe the `main` function. -Page 11, section 5.1.2.2.1, _Program startup_, provides a common definition of `main`: - ----- -int main(int argc, char *argv[]) { /* ... */ } ----- - -The `argc` parameter contains the **c**ount of the arguments provided to the program. -The `argv` parameter contains their **v**alues. - -Create a file named `argv.c` with the following contents: - ----- -#include <stdio.h> - -int main(int argc, char *argv[]) { - for(int i=0; i<argc; i++) { - printf("Argument %d -%s-\n", i, argv[i]); - } -} ----- - -Compile the file running the following command: - ----- -$ cc argv.c ----- - -This produces an executable file named `a.out`. -This executable will print the arguments you provide via the command line: - ----- -$ ./a.out -Argument 0 -./a.out- ----- - ----- -$ ./a.out arg1 arg2 arg3 -Argument 0 -./a.out- -Argument 1 -arg1- -Argument 2 -arg2- -Argument 3 -arg3- ----- - -Note that the first argument is the name of the executable file itself. - -Note that when using quoting, the program produces prints things like: - ----- -$ ./a.out "a b" c -Argument 0 -./a.out- -Argument 1 -a b- -Argument 2 -c- ----- - -So the first argument is `a b` (without quotes). - -=== `exec(3)` - -UNIX-like operating systems provide the `exec` family of functions to invoke commands. -`man 3 exec` describes the `exec` family of functions in Linux. -Linux provides the `execl`, `execlp`, `execle`, `execv`, `execvp`, and `execvpe` functions. -These functions allow us to execute a command from within a C program. - -Create a file named `execlp.c` with the following contents: - ----- -#include <stdlib.h> -#include <unistd.h> - -int main() { - exit(execlp("cat", "cat", "/etc/passwd", NULL)); -} ----- - -Compile the file running the following command: - ----- -$ cc execlp.c ----- - -This produces an executable file named `a.out`. -Execute it: - ----- -$ ./a.out ----- - -This is equivalent to running in a shell the statement `cat /etc/passwd`. - -This article does not describe the intricacies of the `exec` family of functions. -However, let's analyze the call to `execlp`. - -The `exec` functions whose name contains a `p` look up the command to execute by searching for executables named like the first argument in the directories listed in the `PATH` environment variable. -In the example, `execlp` looks up the `cat` executable in directories such as `/usr/bin`. - -The second argument is also the name of the program. - -[NOTE] -==== -Note that in the preceding `argv.c` example, the zeroth argument is the name of the program being executed. - -Some executables in Linux systems are present under different names (using symbolic links). -For example, `xzcat` is a symbolic link to `xz`. -Running `xzcat` or `xz` runs the same executable file, but the executable uses the zeroth argument to change its behavior. - -This technique is a simple way to "share" code between similar programs. -The https://www.busybox.net/about.html[BusyBox] project provides many common utilities, such as `ls` and `cat`, in a single executable. -By sharing code among all utilities, the BusyBox executable is smaller. -==== - -The rest of the parameters to `execlp` are the arguments for the executable file. - -In a way, `exec` functions "call" the `main` function of other programs. -The parameters to `exec` are "passed" to the `main` function. - -=== Shells - -Programs such as `bash` provide a way to execute other programs. -When you type a statement such as `cat /etc/passwd`, `bash` parses the statement into a command to execute and arguments. -Then, `bash` uses an `exec` function to run the program with arguments. - -The simplest `bash` statements are words separated by spaces, of the form `arg0 arg1 arg2 _..._ argn`. - -On such a statement, `bash` executes something like: - ----- -execlp(arg0, arg0, arg1, _..._, argn, NULL) ----- - -And the program will receive the string `arg0` as the zeroth argument, `arg1` as the first argument, and so forth. - -However, using `cat` to view the contents of files, the user might want to view a file whose name contains spaces. - -The statement `cat a b` has two arguments: `a` and `b`. -For each argument, `cat` prints the file of that name. -So the `cat a b` statement prints the contents of the `a` and `b` files, not of a file named `a b`. - -== Further reading - -* http://teaching.idallen.com/cst8177/13w/notes/000_find_and_xargs.html[Using find -exec or xargs to process pathnames with other commands] -* https://infosec.exchange/@david_chisnall/115116683569142801[Early UNIX did glob expansion in the shell not because that’s more sensible than providing a glob and option parsing API in the standard library, but because they didn’t have enough disk space or RAM to duplicate code and they didn’t have shared libraries... For example, on FreeBSD, I often do pkg info foo* to print info about packages that start with some string. If I forget to quote the last argument, this behaves differently depending on whether the current directory contains one or more files that have the prefix that I used. If they do, the shell expands them and pkg info returns nothing because I don’t have any installed packages that match those files. If they don’t, the shell passes the star to the program, which does glob expansion but against a namespace that is not the filesystem namespace. The pkg tool knows that this argument is a set of names of installed packages, not files in the current directory, but it can’t communicate that to the shell and so the shell does the wrong thing. Similarly, on DOS the rename command took a load of source files and a destination file or pattern. You could do rename *.c *.txt and it would expand the first pattern, then do the replacement based on the two patterns. UNIX’s mv can’t do that and I deleted a bunch of files by accident when I started using Linux because it’s not obvious to a user what actually happens when you write mv *.c *.txt. There is a GNU (I think?) rename command and its syntax is far more baroque than the DOS one because it is fighting against the shell doing expansion without any knowledge of the argument structure.] - -== TODO - -* SSH particularities: https://news.ycombinator.com/item?id=36722570[] diff --git a/linux/ssh_for_beginners.md b/linux/ssh_for_beginners.md deleted file mode 100644 index e04b4b97..00000000 --- a/linux/ssh_for_beginners.md +++ /dev/null @@ -1,97 +0,0 @@ -Some simple advice for people who are starting to use ssh. - -# Use the config - -If you create a file `~/.ssh/config`, with contents like: - -``` -Host xxx - HostName yyy - Port 1234 - User zzz -``` - -, then if you type `ssh xxx`, the result will be like executing `ssh -p 1234 zzz@yyy`. - -Any ssh command line arguments can be encoded in an SSH client configuration file, so you can access any server with a simple `ssh host` command, without any additional parameters. - -Additionally, most modern systems configure SSH tab completion, so if you type `ssh <tab><tab>`, your shell will complete with the hosts in your configuration file. - -# Use public key authentication - -By default, ssh uses passwords for authentication. -If you use a good password, then password authentication is a decent authentication method. - -However, you can use other methods, such as public key authentication. -With public key authentication, you have a public and private key. - -If you are working on system A with your *private* key, and you copy your *public* key to system B, then you can ssh from system A to system B without entering a password. - -## Security - -Note that if someone obtains your private key, they will be able to log in to systems that trust your key. -Knowledge of your private key is similar to knowledge of a password. -Take care making your private key truly private. - -If you suspect someone else has been able to obtain your private key, then generate a new key and remove the leaked public key from all systems. - -Note that you can generate as many keys as you want. -Managing multiple keys requires more effort, but in some cases it might be more convenient. -For example, if a key is suspected to be leaked, then you might only need to revoke a key and continue using other keys. - -## Generating SSH keys - -To generate your private and public keys: - -``` -$ ssh-keygen -Generating public/private rsa key pair. -Enter file in which to save the key (/home/alex/.ssh/id_rsa): -Created directory '/home/alex/.ssh'. -Enter passphrase (empty for no passphrase): -Enter same passphrase again: -Your identification has been saved in /home/alex/.ssh/id_rsa -Your public key has been saved in /home/alex/.ssh/id_rsa.pub -The key fingerprint is: -SHA256:... -The key's randomart image is: -+---[RSA 3072]----+ -... -``` - -## Key type choice - -OpenSSH, the standard ssh client, [changed its default type of key generation to Ed25519 in version 9.5 released in late 2023](https://www.openssh.com/txt/release-9.5). -Previously, `ssh-keygen` generated RSA keys, as in the example above. -Many Linux distributions still use OpenSSH versions earlier than 9.5. - -You can find advisories like [the following](https://www.ssh.com/academy/ssh/keygen): - -> It is quite possible the RSA algorithm will become practically breakable in the foreseeable future. All SSH clients support this algorithm. - -Although as of the time of writing this, RSA is considered safe. -However, you can consider generating an Ed25519 key instead, following the most recent OpenSSH defaults. - -## Passphrases - -By default, if you provide an empty passphrase to `ssh-keygen`, your private key will be stored unprotected. -Anyone that can read the private key file can obtain your key. - -You can use a passphrase to protect your key. -If someone obtains a private key file but they don't know the passphrase, then they cannot use the key. - -Using a passphrase means that you need to type the passphrase every time you use the key, or use a system such as `ssh-agent`. -This creates a tradeoff between security and convenience. - -(Note that a popular criticism of SSH public key authentication is that it is not easy for systems administrators to enforce the use of SSH passphrases.) - -# Further SSH features - -Many developers have added many useful features to SSH during many years, such as: - -* The `scp` command to transfer files using SSH -* Tunnels to establish bidirectional communication between systems without such connectivity. - (For example, to connect to your workstation from a remote system.) -* Jump hosts that expedite the connection to a system that is not directly accessible, by using SSH to establish connection through intermediate systems. - -Also, SSH integrates very well with UNIX pipes and tools such as rsync, Git, and many others. diff --git a/linux/using_yggdrasil_opennic_yunohost.md b/linux/using_yggdrasil_opennic_yunohost.md deleted file mode 100644 index 6a144045..00000000 --- a/linux/using_yggdrasil_opennic_yunohost.md +++ /dev/null @@ -1,42 +0,0 @@ -# Using Yggdrasil/OpenNIC/YunoHost - -This is research on doing "gratis" self-hosting. - -* Yggdrasil is an overlay IPv6 network running on top of the Internet. - All devices on Yggdrasil have a fixed IP address, no matter where in the Internet they are. - The Yggdrasil IP address is reachable by other devices in the Yggdrasil network. - - This means that you do not need any public IP address to communicate between devices on Yggdrasil. - - Therefore, you do not need to pay for public IP addresses, and have some extra flexibility. - (You can move a host between networks and you can continue to be reachable without dynamic DNS, etc.) - -* OpenNIC is an alternate DNS root. - Systems using OpenNIC servers can resolve hostnames on OpenNIC. - - Registering OpenNIC domains has no cost. - - Therefore, you do not need to pay a DNS domain. - -* YunoHost provides easy installation of many popular self-hosted services. - -## Notes - -* The Debian README for Yggdrasil sets up a configuration without public peers. - My testing hosts discovered each other only because they were on the same IPv4 network, probably. - -* be.libre domains take a while to be operative. - -## Caveats - -* The OpenNIC ACME service cannot connect to an Yggdrasil host! -* Do not create an initial user on Debian that matches the username you want on YunoHost! (already documented at <https://doc.yunohost.org/en/install_on_top_of_debian>) -* YunoHost seems to override your DNS configuration with a list of public DNS servers (<https://github.com/YunoHost/issues/issues/2444>) -* The YunoHost firewall also messes with Yggdrasil. -* The Debian package for Debian 12 is not compatible with public Yggdrasil nodes. - The backport is good. -* At least be.libre only allows A, AAAA, NS, and TXT records. - YunoHost mail works without MX records, and mail in general should work without MX records. - You can always use NS and host the zone yourself with all the record types you need. -* Let's Encrypt does not issue OpenNIC certificates, and the only alternative seems to be an experimental CA that supports ACME. - The OpenNIC CA does not connect to Yggdrasil, but you could host your zone yourself and use a DNS server friendly to ACME. diff --git a/misc/aws/account_setup_notes.md b/misc/aws/account_setup_notes.md deleted file mode 100644 index 46647fe1..00000000 --- a/misc/aws/account_setup_notes.md +++ /dev/null @@ -1,88 +0,0 @@ -# Account setup notes - -> [!CAUTION] -> I do not have AWS training nor significant experience. -> The information here can be insecure and dangerous. - -## Initial setup - -### Creating the initial management account and initial configuration - -Opening an account requires a credit card, a phone number and an email address. - -Once an email address is associated with an AWS account, no other AWS account can be created with the same email address, even if you delete the AWS account. -However, you can change the email address of the AWS account to "free" the email address. - -Create use multiple email addresses or plus addressing for experiments, etc.. -(OVH "redirect", Gmail and Google Groups support plus addressing.) - -Go to "IAM Identity Center" and enable it in your preferred region. -This enables AWS Organizations and creates a `Root` OU that contains the management account. - -By default, this uses an internal "Identity Center directory". -Alternatively, you can integrate with an external identity provider. -(TODO: using SAML?) - -Create an `Admins` group. - -Create an `admin` user, adding the user to the `Admins` group. - -Create a predefined permission set with the `AdministratorAccess` policy. - -Go to multi-account permissions, AWS accounts, select the management account and click "assign users or groups". -Assign the `Admins` group with the `AdministratorAccess` permission set. - -Note the "AWS access portal URL", users use this URL to log in. - -### Setting up the `admin` user - -Depending on how you created the user, you create the password by following an email link or you receive an initial password. - -You have to set up MFA. - -When the user signs in, they are redirected to the AWS access portal. -An account tab displays the AWS accounts that the user can access. -Expand the account and click `AdministratorAccess` to access the AWS Console with full administrator access. - -#### Configuring `awscli` - -``` -$ aws configure sso -SSO session name (Recommended): ${enter something} -SSO start URL [None]: ${the AWS access portal URL from an earlier step} -SSO region [None]: ${enter one} -SSO registration scopes [sso:account:access]: ${leave blank} -``` - -The `configure sso` command prints: - -``` -aws s3 ls --profile ${your profile name} -``` - -You can use this command to test access. - -To log in again: - -``` -aws sso login --profile ${the profile name from an earlier step} -``` - -## First steps with AWS Organization Formation - -``` -npx --package aws-organization-formation -- org-formation init ${starter template yaml} --profile ${the profile name from an earlier step} -``` - -This command creates a `${starter template yaml}` file with the skeleton of your current AWS account structure. - -## Account closure - -TODO: verify when deleting an account disposes resources that incur billing. - -## References - -* [AWS Organization Formation](https://github.com/org-formation) declarative management of AWS accounts - * [org-formation-reference](https://github.com/org-formation/org-formation-reference) A reference architecture which aims to provide some best practices for any AWS Organization starting out using org-formation. - * [Part 1 – Managing AWS accounts like a PRO](https://fourtheorem.com/managing-aws-accounts-part-1/) - * [Part 2 – Managing accounts using IaC with OrgFormation](https://fourtheorem.com/managing-accounts-using-iac-and-orgformation/) diff --git a/misc/document-formats.md b/misc/document-formats.md deleted file mode 100644 index 9ee21acc..00000000 --- a/misc/document-formats.md +++ /dev/null @@ -1,83 +0,0 @@ -# Document formats - -Most of the time, when writing a document, I want a document format with the following properties: - -* Fast to write using a plain text editor -* Easy to parse into an AST - -An AST is a programming-friendly representation of a document. -ASTs reduce the effort required to write tools such as a program that validates links in a document. -Ideally, ASTs contain information to track a document element to the position it occupies in the original document. -With this information, if you write a tool such as a spell checker, then you can highlight misspelled works precisely in the original document. - -On top of that, some features that I don't always need: - -* Math support -* Sophisticated code blocks. - For example, being able to highlight arbitrary parts of code blocks (not syntax highlighting). -* Diagram support - -## Existing formats - -### Markdown - -* Easy to write using a plain text editor -* Has good AST parsers with position information -* Has math support -* Does not support sophisticated code blocks -* There are many extensions with support for math, diagrams, and many others -* Is very popular and supported everywhere -* However, there is a wide variety of variants and quirks -* Especifically, because Markdown was not designed with parsing in mind, so tools based on different parsers can have differences in behavior - -### [Djot](https://djot.net/) - -It is very similar to Markdown, except: - -* It is designed for parsing, so independent parsing implementations are very compatible with each other -* It is not so popular, so there are less extension and tool support - -### [AsciiDoc](https://asciidoc.org/) - -Compared to Markdown: - -* It's more complex to write, but mostly because it's different and more powerful -* There are attempts to write better parsers, but good parsers with position information are not available yet -* Supports sophisticated code blocks -* It has a smaller ecosystem than Markdown, but many good quality tools such as Antora - -### [Typst](https://typst.app/) - -Checks all my boxes, except: - -* It is designed for parsing and it has an AST, but it is not easy to access -* Currently Typst is very oriented towards generating paged documents (e.g. PDF) -* It includes a full programming language, which is mostly good (very extensible), but this might increase complexity undesirably - -Typst is very new and is not yet very popular. - -[Typesetter](https://codeberg.org/haydn/typesetter) is a desktop application that embeds Typst, so no additional setup is needed. -However, Typesetter is only available as a Flatpak. - -### [Verso](https://github.com/leanprover/verso) - -A Markdown-like closely tied to [the Lean programming language](https://lean-lang.org/): - -* Eliminates ambiguous syntax for easier parsing and is stricter (not all text is valid Verso) -* Has a (Lean) data model -* Designed for extensibility - -### TODO: other formats - -- https://github.com/nota-lang/nota -- https://github.com/christianvoigt/argdown -- https://github.com/nvim-neorg -- https://github.com/podlite/podlite/ -- https://orgmode.org/ -- https://github.com/sile-typesetter/sile - -## Creating your own formats - -https://github.com/spc476/MOPML someone created its own lightweight format using Lua and PEGs. - -https://tratt.net/laurie/blog/2020/which_parsing_approach.html has information about choosing parsing approaches. diff --git a/misc/internet-communication-channels.md b/misc/internet-communication-channels.md deleted file mode 100644 index 7ba95f0e..00000000 --- a/misc/internet-communication-channels.md +++ /dev/null @@ -1,203 +0,0 @@ -# Internet communication channels - -If you want to provide a communication channel for a community over the Internet and you are considering options such as: - -* Slack -* Discord -* Reddit -* Telegram -* WhatsApp -* Facebook -* Or any other communication channel controlled by a single big company - -, then please read this article and consider an alternative. - -Because such channels are often convenient, cheap, and easy, they are natural choices. - -However, companies are about maximizing their benefits first. -Certainly, providing convenient, cheap, and easy services often help companies make money. -But I believe we have seen enough examples of companies putting their benefits first in detriment of their users. - -Using these alternatives will always require more effort. -This text is long, and just reading and processing it might take more time than setting up a channel on the services mentioned above. -The alternatives I describe certainly have drawbacks compared to the services I am asking you to avoid. -However, in the long run I think making an extra effort to make an informed choice pays off. - -## A quick litmus test - -If you only thing about a single thing, then think about this: how many independent clients are for this communication channel? - -How tightly the people behind the channel control clients is a good indicator of how much they want to maximize profits. - -## Alternatives - -### Instant messaging - -#### IRC - -IRC is a real-time chat protocol created in 1988 that is still in use. -Many perceive flaws in IRC that seem to make it a bad choice. -However, many IRC flaws have been addressed in recent times and I believe it is a good choice in many (but not all) scenarios. - -The biggest traditional issue with IRC is channels without history, where you cannot see messages posted while you were offline. -(If you suspend or turn off your laptop, you will be offline in IRC. -Even if you run your IRC client continuously on your client, if your phone goes out of coverage or your phone suspends your IRC client, you will be offline.) -However, nowadays you can create channels with history. - -Channels without history are frequently confusing for new users, because most chat systems have history. -Heavy IRC users are either used to having no history [this might seem surprising, but for some this is even a benefit] or have means to be permanently connected to IRC. -However, users new to IRC might join a channel, post a question and go offline without anyone having a chance to see their message and reply. -Then, unless people remember to answer when they are back, or other means are used, answers will not be visible to the person who asked. - -[The `chathistory` extension](https://ircv3.net/specs/extensions/chathistory) addresses this problem. -As far as I know, only [the Ergo IRC server](https://ergo.chat/about) and [their network](https://ergo.chat/about-network) support this extension. - -Some advantages of IRC are: - -* You can use IRC without creating an account. - This can be especially useful for providing a general contact mechanism. - You can create links that will ask for a nickname, and place you into a channel without any additional steps. - -* IRC is a very simple protocol with more than 30 years of history. - This means that many developers have invested significant efforts in creating powerful IRC clients and tools (such as bots). - And lately, many easy IRC clients are available. - This means that IRC can scale from simple setups that require little effort to use, to powerful setups that can provide interesting features. - (If you are used to plain communication clients, you might be surprised at how valuable some features can be.) - -Some drawbacks of IRC are: - -* IRC does not have end-to-end encryption, and thus IRC administrators can read every conversation. - This is not a huge problem for public or semi-public channels, but it limits IRC for many scenarios. - -* IRC requires more effort from administrators to provide a good experience to entry-level users, control spam, and others. - (An important point is that although integration with audio/video conferencing is possible, it requires more effort and provides a lesser experience.) - -* IRC is mostly text-based. - Although many IRC clients can display images and GIFs, communicating with images and GIFs is harder on IRC. - (And IRC also does not have integrated audio/video conferencing.) - -* Push notifications are not common yet. - Although it is possible to receive instant notifications when you are mentioned or receive a private message, this is frequently difficult. - In general, IRC on mobile phones is not as evolved as on desktop computers. - -Interesting projects: - -* Web clients - * https://github.com/ObsidianIRC/ObsidianIRC - * https://kiwiirc.com/ - * https://codeberg.org/emersion/gamja -* https://soju.im/ (Bouncer) -* https://halloy.chat/ (Graphical desktop client) -* https://git.sr.ht/~delthas/senpai (Terminal client) - -### Delta Chat - -Delta Chat is an instant messaging system that tries to be very similar to the most popular instant messaging systems. - -However, there are multiple clients and anyone can run a server. - -The strangest thing about Delta Chat is that is uses email underneath. -However, I would recommend ignoring this fact. - -#### XMPP - -XMPP is younger than IRC, but older than Matrix. -Compared to Matrix: - -* End-to-end encryption and audio/video conferencing is possible with XMPP, but in practice it can be difficult to access these features. - -* There's more XMPP clients than Matrix clients, but it is also hard to find clients that support all the features you need on different platforms. - -For some scenarios, if you find the right combination of XMPP server and clients, XMPP can be a great option. - -Historically, XMPP was not well-suited to mobile usage. -Nowadays, mobile usage is better, but finding the right clients to use is still a challenge. - -#### Matrix - -Matrix is a more modern chat protocol that addresses some of the drawbacks of IRC: - -* Matrix has end-to-end encryption, so conversations between users are private to Matrix administrators. - -* Matrix requires less effort from *channel* administrators. - (But running a Matrix server requires significant resources. - However, there are public Matrix servers and managed services. - Thanks to end-to-end encryption, using a public Matrix server is an interesting option.) - -* Matrix has good support for audio/video conferencing, images and GIFs, reactions, push notifications, and phone usage. - -But also some disadvantages compared to IRC: - -* Users need to create accounts. - -* Using end-to-end encryption makes some usage harder. - (Although end-to-end encryption is optional.) - -* There are fewer clients and tools, and generally they are more complex, more resource intensive, and less featureful. - (And not all clients support all features.) - -#### Other instant messaging alternatives to consider - -* Zulip: Zulip offers instant messaging, but has some characteristics from forums. - (For example, Zulip uses threads with subjects.) - -* Mattermost, Rocketchat are designed for communication within organizations. - -And lastly, because all the technologies mentioned in this text allow integrations, there are bridges to join different technologies. - -For example, IRC channels can be bridged to Matrix rooms. - -Although bridges are not ideal, in some cases you can use them to make one channel available over different technologies, which might address the limitations of specific technologies. - -### Asynchronous messaging - -Although my perception is that most communities nowadays communicate over instant messaging, many communities use successfully more asynchronous communication channels. -In some cases, providing both instant messaging and an asynchronous channel can also work well. - -#### Mailing lists - -Mailing lists (and their sibling, newsgroups) are older than IRC. -Although mailing lists are far less popular than in the past, many communities still use mailing lists. - -Mailing lists have several advantages: - -* Having an email address is nearly a necessity for all Internet users. - Mailing lists often require no user account other than an existing email address. - -* In a way, email and mailing lists share many similarities with IRC. - Although most people are users of just a few mail services and clients, there is a wide variety of services and clients. - Email power features are somewhat forgotten, but they still exist and mail clients can have very convenient features. - -* Most mailing list have good ways to browse and search past messages. - Email discussions are more naturally searchable, thanks to their slower pace and thread organization. - -However, they also have many advantages: - -* As people no longer use email to communicate, going back to email can cause significant friction. - -* Finding a good mailing list service is difficult. - (And hosting your own is also more difficult than hosting other services.) - -In my opinion, mailing lists are good, but they have become foreign to most people. - -#### Web forums - -Forums used to be very popular. - -Compared to mailing lists: - -* Forums require creating an account. - -* Forums do not have multiple clients, although forum software has also evolved for a long time, and many forums have great features. - -* Forums are also a bit out of style, but they are more popular and familiar to most than mailing lists. - -* Finding a forum service or hosting one is simpler than email. - -### Other possibilities - -Social networks tend to be slightly different communication channels than instant messaging or asynchronous messaging. -Alternatives to social networks also exist. -However, in my opinion, social network-style communication is not optimal for "communities" in most cases. -Still, you might want to explore alternatives. -The Fediverse (or ActivityPub) has many different varieties of communication channels that might suit your needs. diff --git a/misc/problemas.md b/misc/problemas.md deleted file mode 100644 index c8da58f6..00000000 --- a/misc/problemas.md +++ /dev/null @@ -1,91 +0,0 @@ -# Problemas - -Este documento es una lista de cosas que me tocan las narices. -En el mundo hay infinidad de problemas más graves, pero quiero destacar esta lista. - -## La basura telefónica está fuera de control - -Recibo frecuentemente tanto llamadas como SMS fraudulentos o de publicidad indeseada. -Estoy apuntado en la lista Robinson y simplemente no hay manera de librarse. - -No corro un riesgo severo de ser víctima de un fraude, pero imagino que una cantidad importante de gente sí lo corre. -(Aunque una vez, la compañía eléctrica de la que era cliente sí me coló un timo.) - -Hasta donde yo sé: - -* Es trivial falsear la identificación de un SMS para que sea idéntico al que envía una entidad legítima. - (E.g.: puedes hacer un SMS que se identifique como "Correos", igual que un SMS legítimo de Correos.) -* No existe, hasta donde yo sé, todo mecanismo de denuncia requiere que identifiquemos al autor de la llamada. - Identificar quién llama sólo es posible si la empresa decide identificarse. -* No percibo ninguna consecuencia negativa para nadie que participe en la basura telefónica. - Esto incluye a quienes realizan estas llamadas y SMS, y a las operadoras telefónicas por las que fluyen. - -Los filtros de Google son moderadamente efectivos, pero no están al alcance de todo el mundo. -Además, los falsos positivos de los filtros pueden hacer perdernos comunicaciones legítimas importantes. - -### Recomendaciones frente a la basura telefónica - -A nivel individual, podemos formarnos para hacernos menos vulnerables a los fraudes, pero es prácticamente imposible evitar las molestias. - -Mi recomendación para quien tenga un móvil con posibilidad de reportar llamadas de spam, es coger las llamadas que muestren un número y: - -* Decir algo. - Si no decimos nada, muchas centralitas de spam no conectan a un operador, con lo que no podemos tener 100% la certeza de que sea spam. -* Esperar a que respondan para asegurarnos de que es spam. -* Si parece que hay un robot al otro lado de la línea, colgar inmediatamente. - Si parece que hay un humano, esperar a que cuelgue. - (Con esto, el humano no está libre para hacer otra llamada, con lo que les frenamos un poco. - También podemos intentar alargar la llamada, aunque yo personalmente no tengo paciencia.) -* Marcar la llamada como spam. - -Tengo la sensación de que los indicadores de spam de muchos teléfonos se basan en las denuncias que recibe cada número. -Así que cuantas más llamadas se cojan y se marquen como spam, antes aparecerán marcadas claramente como spam para otras personas. - -(Es importante que los filtros antispam sean precisos.) - -### Otras referencias - -* [Hiya Global Call Threat Report Q4 2024](https://www.hiya.com/global-call-threat-report), que redirige a [este PDF](https://6751436.fs1.hubspotusercontent-na1.net/hubfs/6751436/Global%20Call%20Threat%20Report_2024Q4.pdf). - -## Los protocolos cerrados dan un poder desproporcionado a empresas privadas - -Para la mayoría de gente, WhatsApp es prácticamente una necesidad para la vida cotidiana. - -Esto hace que Meta controle una parte sustancial de nuestras comunicaciones, queramos o no. -Además, esto hace que cualquier problema con WhatsApp (incidencia, carencia, etc.) sea inevitable. - -También hay efectos inesperados como que Meta decide qué sistemas operativos móviles son viables y cuáles no. -(En un par de ocasiones, me he tenido que cambiar de móvil porque Meta ha decidido dejar de soportarlo. -Aunque puedo experimentar con sistemas operativos móviles alternativos, siempre tengo que tener un móvil soportado por WhatsApp.) - -[La ley de mercados digitales (DMA)](https://maldita.es/malditatecnologia/20240313/dma-ley-mercados-digitales-usuarios/) en teoría ayudará parcialmente. -Esta ley debería obligar a WhatsApp a interoperar, con lo que podríamos comunicarnos con usuarios de WhatsApp sin usar WhatsApp, mitigando algunos problemas. -Sin embargo, aunque lleva en vigor desde el 7 de marzo de 2024, esto todavía no es posible y está por ver cuán efectivo será. - -Muchos sistemas de comunicación existentes son más abiertos que los protocolos modernos: - -* Cualquiera puede montar un servidor de correo y comunicarse con usuarios de correo electrónico de otros proveedores. - Pese a que muchos apuntan a que Google y Microsoft tienen un poder desproporcionado de facto, sigue siendo totalmente viable usar otros proveedores. - Y aunque se apunta que la interoperabilidad de los correos es causante del spam, muchos otros sistemas cerrados como WhatsApp tienen problemas de spam similares o mayores. - -* Aunque no todo el mundo puede hacer emisiones de DVB-T, cualquiera con una antena puede captar las emisiones y visualizarlas. - (Los protocolos con los que se codifican las emisiones de DVB-T están disponibles para cualquiera.) - También es posible codificar las emisiones de DVB-T para limitar su uso a usuarios que paguen, pero con libertad de consumir los contenidos con cualquier sistema DVB-T de nuestro agrado. - -(Esto en contraste con los servicios de streaming, que sólo podemos usar con dispositivos validados por el servicio de streaming.) - -## Los navegadores son excesivamente complejos - -Gran parte de los contenidos y procesos que tenemos que realizar hoy en día pasan por un navegador web. - -Por diversos motivos, los navegadores cada vez son más sofisticados para permitir mayores funcionalidades. -Son tan complejos que Microsoft, una de las mayores empresas tecnológicas del mundo, ha renunciado a desarrollar un navegador propio y reutiliza gran parte de Chrome, un navegador controlado por una empresa con la que compite, Google. - -Fuera de Google Chrome y de Safari de Apple, virtualmente no existen navegadores que compitan con ellos. -(Hay más navegadores, pero como Edge de Microsoft, usan el motor de Chrome o de Safari. -Firefox es cada vez más minoritario e irrelevante [aunque yo lo uso y animo a todo el mundo a que lo use].) - -La sofisticación y complejidad de Chrome y Safari adicionalmente hacen que cada vez existan más webs y aplicaciones web que son prácticamente inutilizables en dispositivos de rendimiento modesto. -Esto hace que sea virtualmente necesario renovar nuestros dispositivos con más frecuencia de la necesaria, a dispositivos más costosos de lo que necesitaríamos para el resto de nuestros propósitos. - -He escrito más sobre el tema en inglés en [a plan against the current web](../programming/a_plan_against_the_current_web.md) y [the content web manifesto](../programming/the-content-web-manifesto/README.md). diff --git a/misc/ripping.md b/misc/ripping.md deleted file mode 100644 index f985c43f..00000000 --- a/misc/ripping.md +++ /dev/null @@ -1,71 +0,0 @@ -# Ripping - -## Media - -[Main source](https://arstechnica.com/civis/threads/ripping-optical-media.1507399/post-43734994). - -### Audio CD - -About 200-300 MB per album CD when ripped to FLAC. - -### DVD - -About 4-8 GB per disc, averaging 5.6 GB per movie as ISO. - -### Blu-ray - -About 20-50 GB per disc, averaging 37 GB per movie as ISO. - -## Hardware - -### Reader - -I got a Verbatim external USB Blu-ray writer for about 120€. - -### Storage - -See <https://diskprices.com/>. - -## Software - -### Audio - -* [abcde](https://abcde.einval.com/wiki/) claims to rip and compress to FLAC and tag automatically. - -### Video - -#### DVD - -Use `dd` to rip DVD. -However, `dd` can fail on some disks, perhaps due to damage or copy protection. -[This post on unix.stackexchange describes a trick that works](https://unix.stackexchange.com/a/642790): - -* Start playback of the disc using [VLC media player](https://www.videolan.org/vlc/). -* Try `dd` first, if it fails, then run a command like `ddrescue -n -b2048 -K1M /dev/sr0 x.iso x.map`. -* After `ddrescue` starts running, quit VLC. - -For playback, most software (including Kodi and VLC for Android) can play back DVD ISO with full menu support - -#### Blu-ray - -[FindVUK](http://fvonline-db.bplaced.net/) has the keys to play Blu-ray discs ripped with `dd`. -However, with encrypted Blu-ray discs, you need to configure the keys in each device where you want to play back the content. -(And this is not easy or possible in some cases.) - -[blu-save](https://git.sr.ht/~shironeko/blu-save) can remove the encryption. -Remember to specify the path to the keys when running blu-save. - -However, VLC is confused by the `AACS` and `CERTIFICATE` directories that blu-save copies to the output. -If you remove them, then VLC can play the `BDMV` directory with menus, etc. - -You can repack a Blu-ray extracted with blu-save by running a command like: - -``` -mkisofs -allow-limited-size -o .../my.iso . -``` - -from the directory that contains *only* the `BDMV` directory. - -VLC for desktop computers can open a repacked Blu-ray ISO and show the menus. -Kodi for Android can open a repacked Blu-ray ISO and identify the titles. -However, Kodi did not support the menus for the Blu-ray I tested. diff --git a/misc/take-the-less-traveled-road.md b/misc/take-the-less-traveled-road.md deleted file mode 100644 index be5fb72b..00000000 --- a/misc/take-the-less-traveled-road.md +++ /dev/null @@ -1,32 +0,0 @@ -# Take the less traveled road - -> Two roads diverged in a wood, and I—\ -> I took the one less traveled by,\ -> And that has made all the difference. - -(Robert Frost, [The Road Not Taken](https://www.gutenberg.org/cache/epub/59824/pg59824-images.html#THE_ROAD_NOT_TAKEN)) - -[Infrastructures](https://xkcd.com/743/), by Randall Munroe - -The prisoner's dilemma describes situations where people can choose to act in their own interest or collaborate with others. -I believe the prisoner's dilemma is present everywhere in our daily life and can explain many behaviors and situations. - -One such scenario is when we choose services to use on the Internet. - -Frequently, we choose services that are easy, popular, and even have no cost. - -In the past, even non-technical people got shared hosting and ran WordPress to blog, with nearly total control about their communication. -Nowadays, mostly everyone uses something like Twitter and cedes control to a company. - -Using Twitter is easier than self-hosting WordPress. -Twitter is so popular, that your message will likely reach more users on Twitter than on an independent blog. -And you can use Twitter for free, whereas shared hosting costs you money and time. - -Twitter became dominant and nowadays, many are frustrated by how Twitter has changed. - -Dominance implies lack of competition, which is nearly always bad for the consumer. - -Which means choosing which services to use frequently is a difficult choice. - -We all have limited time and energy, but I propose that whenever you can, you should take the less traveled road. -The easy alternative likely benefits you in the short term, but can easily contribute to a monoculture that will damage everyone in the end, including you. diff --git a/programming/a_plan_against_the_current_web.md b/programming/a_plan_against_the_current_web.md deleted file mode 100644 index 55e58da4..00000000 --- a/programming/a_plan_against_the_current_web.md +++ /dev/null @@ -1,19 +0,0 @@ -# A plan against the current web - -Browsers are controlled by Google, and to a lesser extent, Apple. -We have arrived to this because browsers have become excessively complex, so that even Microsoft has decided that it's better to be subject to the whims of Google, than to maintain one. -This complexity is derived from using the web for delivering applications, where its initial purpose was browsing content. - -## Part I: Make content websites simple again - -See [the content web manifesto](the-content-web-manifesto). - -## Part II: Application distribution sucks - -We use so many web applications nowadays because there are no alternative platforms to distribute applications with the same reach and convenience. - -As a way to start this discussion, let's propose making Flatpak applications work on Windows and macOS, and make them installable and executable from a web link (like Java Web Start or ClickOnce from Microsoft). -And additionally, let's produce "starter packs" for as many programming languages as possible, so creating these applications is "easy". - -Besides all the implementation work, what would be the downsides to this? -I believe that this would offer better performance than webapps (Flatpak applications on Linux are consistently faster than web apps and Electron apps), and Flatpak apps can already be implemented using many programming languages (webapps are halfway there through WASM, but not there yet). diff --git a/programming/git/combining_repos_with_josh_filter.md b/programming/git/combining_repos_with_josh_filter.md deleted file mode 100644 index d63506a3..00000000 --- a/programming/git/combining_repos_with_josh_filter.md +++ /dev/null @@ -1,252 +0,0 @@ -# Combining repos with josh-filter - -## Introduction - -When writing complex software, developers frequently consider incorporating code from other repositories into their repository. -This choice is controversial and this document will not discuss whether incorporating other repositories is the best option. - -Developers have different options to perform this task, including: - -* Copying the code -* Using Git submodules -* Using Git subtree -* Using repository "orchestration" tools such as Google's repo - -A recent option is Josh. -Josh is a daemon that can serve virtual Git repositories that apply transformations to repositories. -With Josh, you can create a virtual Git repository that combines multiple repositories, or a virtual repository that contains a subtree of another repository. - -Because Josh is a daemon, using Josh has a greater overhead than other options to combine code from other repositories. -However, Josh provides the josh-filter command that can be used for similar purposes without the extra maintenance of a service. - -This document describes a sample scenario and a step-by-step procedure that you can follow along to learn about josh-filter. - -## Scenario - -A development team maintains the `foo` repository. -The code in the `foo` repository uses external code from the `bar` repository. -The team needs to make changes to code in the `bar` repository, but they have not found a convenient procedure to do so. -Ideally, the team would like to synchronize their changes with the `bar` repository, so that they can benefit from `bar` updates, and contribute back. - -## Preparing the example - -Create an `example` directory to contain all the files required for this example. - -To follow this example, you will need two repositories standing for the `foo` and `bar` repositories. -You can use any repository, but the example assumes that the repos are called `foo` and `bar`, and that the `bar` repository will be copied to the `external/bar` path in the `foo` repository. - -The example works with two local mirrored repositories in Git, so you can simulate pushing and pulling from a Git provider such as GitHub. -The example uses `foo.git` and `bar.git` as the URLs of the two repositories. -You can replace the URLs with real repository URLs, or you can create these repositories by mirroring real repositories. -If you use local mirrors, then you can also simulate pushing and pulling without affecting real repositories. - -To mirror two repositories locally: - -``` -git clone --mirror $URL_TO_SOME_REPO foo.git -git clone --mirror $URL_TO_SOME_REPO bar.git -``` - -(The example also assumes that you are using branches named `main` in both repositories.) - -You also need the josh-filter tool. -Follow [the installation instructions](https://josh-project.github.io/josh/reference/cli.html#josh-filter). - -## Walkthrough - -### Incorporate a repository - -Start by cloning the `foo` repository: - -``` -git clone foo.git -``` - -This command clones the `foo.git` repository to the `foo` directory. - -Change to the `foo` clone. - -``` -cd foo -``` - -Create and switch to a `incorporate-bar` branch. - -``` -git switch -c incorporate-bar -``` - -Fetch the `main` branch of the `bar` repository. -The `get fetch` command creates a `FETCH_HEAD` reference that contains the `main` branch. -This command is a convenient way to work with multiple repositories. - -``` -git fetch ../bar.git/ main -``` - -Use the `josh-filter` command to incorporate the code in the `FETCH_HEAD` reference to the `external/bar` path. -This command takes the `HEAD` reference as an input, and filters another reference using a josh filter. - -The following command takes the `FETCH_HEAD` reference that contains the `bar` code. -The `:prefix=external/bar` moves all content of the `FETCH_HEAD` reference to the `external/bar` path. -The result is stored in a new `FILTERED_HEAD` reference. - -``` -josh-filter ':prefix=external/bar' FETCH_HEAD -``` - -Merge the `FILTERED_HEAD` reference into your current branch. -Because the `FILTERED_HEAD` reference contains the `bar` code and is unrelated to the current branch in the `foo` repo, you need the `--allow-unrelated` option. - -``` -git merge --allow-unrelated FILTERED_HEAD -``` - -After this command, `ls external/bar` and `git log external/bar` show the contents and history of the `bar` repository. -`git log` and tools such as `gitk` will show the combined history of the two repositories. - -Push the branch to the `foo.git` remote. - -``` -git push --set-upstream origin incorporate-bar -``` - -If you were working with a real repository, then you could create, review, and merge a pull request by following the usual procedures. -If you are using mirrored repositories, then change to the main branch and merge the `incorporate-bar` branch. - -``` -git switch main -git merge --no-ff incorporate-bar -git push -``` - -(`git merge --no-ff` is equivalent to the "create a merge commit" button in PRs for GitHub.) - -At this point, the `main` branch in the `foo.git` repository contains the code from the `main` branch in the `bar.git` repository in the `external/bar` path. -The code has the full history, and changes can be contributed to and from the `bar.git` repository. - -### Incorporating upstream changes from the `bar.git` repository - -If new changes are pushed to the `bar.git` repository, then you can pull those changes into the copy in the `foo.git` repository. - -#### Simulating changes in the `bar.git` repository - -Change to the `example` directory. - -Clone the `bar.git` repository. - -``` -git clone bar.git -``` - -Change to the `bar` directory and make some changes. - -Push the changes to `bar.git`. - -``` -git push -``` - -#### Incorporate the changes - -Change to the `example/foo` directory. - -Create and switch to a `pull-bar` branch. - -``` -git switch -c pull-bar -``` - -Fetch the changes again. - -``` -git fetch ../bar.git/ main -``` - -If you run the `git log FETCH_HEAD` changes, then you can verify that the changes you made are in `FETCH_HEAD`. - -Filter, merge, and push the changes again. - -``` -josh-filter ':prefix=external/bar' FETCH_HEAD -git merge --allow-unrelated FILTERED_HEAD -git push --set-upstream origin pull-bar -``` - -After these commands, the `pull-bar` branch contains the new changes from `bar.git`. - -If you were working with a real repository, then you could create, review, and merge a pull request by following the usual procedures. -If you are using mirrored repositories, then change to the main branch and merge the `pull-bar` branch. - -``` -git switch main -git merge --no-ff pull-bar -git push -``` - -### Upstreaming changes in `foo.git` to `bar.git` - -If you make changes to the `external/bar` directory in the `foo.git` repository, you can contribute these changes back to the `bar.git` repository. - -#### Simulating changes in the `foo.git` repository - -Change to the `example` directory and to the `foo` directory. - -Make some changes to the `external/bar` directory. - -Push the changes to the `foo.git` repository. - -``` -git push -``` - -### Upstreaming the changes - -Change to the `example` directory. - -Change to the `bar` directory. - -``` -cd bar -``` - -Create and switch to a `upstream-from-foo` branch. - -``` -git switch -c upstream-from-foo -``` - -Fetch the `main` branch from the `foo.git` repository. - -``` -git fetch ../foo.git/ main -``` - -Apply the opposite filter. -The `:/external/bar` filter puts the contents of the `external/bar` directory in the root of the repository. - -``` -josh-filter ':/external/bar' FETCH_HEAD -``` - -``` -git merge --allow-unrelated FILTERED_HEAD -``` - -After these commands, the `upstream-from-foo` branch in the `bar.git` repository contains the upstream changes from `foo.git`. - -If you were working with a real repository, then you could create, review, and merge a pull request by usual procedures. -If you are using mirrored repositories, then change to the main branch and merge the `upstream-from-foo` branch. - -``` -git switch main -git merge --no-ff upstream-from-foo -git push -``` - -## Further possibilities - -This walkthrough explains how to incorporate code from a repository into a different repository. -Then, you can synchronize further changes to both repositories. - -With similar steps, you can experiment how other Git functionality is affected, such as resolving merge conflicts, or different Git workflows. diff --git a/programming/git/git_advice.md b/programming/git/git_advice.md deleted file mode 100644 index 7dd19818..00000000 --- a/programming/git/git_advice.md +++ /dev/null @@ -1,22 +0,0 @@ -# Git advice - -## Never use `git commit -m`, use `git commit -v` - -Configure your system so that `git commit` opens your preferred editor. - -With `git commit -v` you can see your commit diff while writing your commit message. -This helps you review that your commit is correct and write a better commit message. - -## Use gitignore properly - -See <https://git-scm.com/docs/gitignore>. - -Note that by default, Git defaults to `$XDG_CONFIG_HOME/git/ignore` or `$HOME/.config/git/ignore`. - -## Use the modern Git commands (or teach them) - -Particularly, `git checkout` has many functionalities that now can be handled by more focused commands like `git switch` and `git reset`. - -If you have too much muscle memory and are used to them, then consider learning them only to teach other people so that they start with the safer commands. - -Many Git commands print suggestions that use the newer commands. diff --git a/programming/git/github_annoyances.md b/programming/git/github_annoyances.md deleted file mode 100644 index df373bd7..00000000 --- a/programming/git/github_annoyances.md +++ /dev/null @@ -1,9 +0,0 @@ -# GitHub annoyances - -## The repository creation wizard can be confusing initially - -When creating a new repo, GitHub offers you to populate the repository with some files (a README, a `.gitignore` file, a license). - -In some situations, you have an existing directory in your computer with files that you want to be the initial contents of the repo. -If you create a truly empty repo, then GitHub displays some instructions that can help pushing the contents of your existing directory to the new repo. -If you use the GitHub features to populate the repo, then GitHub does not display these instructions and uploading your files requires more knowledge. diff --git a/programming/java/tutorial.md b/programming/java/tutorial.md deleted file mode 100644 index e4f5f1d1..00000000 --- a/programming/java/tutorial.md +++ /dev/null @@ -1,84 +0,0 @@ -# A Java tutorial - -This tutorial walks through creating a blank Spring Boot application on Pop!_OS 22.04 - -## Set up - -Open a terminal and run: - -``` -$ sudo apt install openjdk-21-jdk code -``` - -and follow the prompts. - -(You can install both packages from the Pop!_Shop application. -However, when installing Visual Studio Code, Pop!_Shop defaults to the Flatpak version of the program that is more troublesome than the .deb package. -Choosing the correct software is easier from the terminal.) - -Open Visual Studio Code from "Show Applications". - -Follow the "Walkthrough: Setup VS Code", but in the "Rich support for all your languages" step, click "Browse Language Extensions" and install "Extension Pack for Java". -Do not install a new JDK; you installed one in a previous step. - -## Creating a Spring Boot application - -In the command palette (ctrl+shift+p), search for and execute "Java: Create Java Project...". - -Select "Spring Boot". - -Install the Initializr plugin if prompted, you might need to restart the Java project creation wizard. - -Select "Maven Project", select the highest Spring version that is not a snapshot. - -Select the Java language. - -Enter a group id. A group should be a domain in reverse order. The default of "com.example" is OK. - -Enter an artifact id. This should be a single keyword. The default of "demo" is OK. - -Select the jar packaging type. - -Select the Java 21 version matching the JDK you installed in a previous step. - -Add only the Spring Web dependency. - -When choosing the folder for the project, Visual Studio Code creates a further folder named like the artifact id you enter in a previous step. -(So do not create a directory with your application name.) - -Choose File, Open Folder in the Visual Studio Code menu, then select the directory that the previous step created named like the artifact id. - -"Trust the authors". - -Navigate to the `src/main/java/com/example/demo/DemoApplication.java" file (the path varies depending on the group id and artifact id). - -Right click on the file and select "Run Java". -If the "Run Java" option is not present, then you might not have "trusted the authors"; in this case, Open Folder again. - -Visual Studio Code displays a terminal. -After a few moments, the terminal displays a message about the application having started. - -Open a browser and navigate to <http://localhost:8080>. -The browser displays an error because the application wizard creates an empty application. - -## Advice for people involved in developing the software used in this tutorial - -### The Eddy Pop_OS! tool should allow interacting with debconf - -When using the Visual Studio Code .deb from Microsoft and not the .deb from Pop!_OS, installing the package from the browser fails. -The Visual Studio Code .deb has a debconf prompt to add Microsoft package repositories. -This locks Eddy. - -This is likely covered by these issues: - -* https://github.com/donadigo/eddy/issues/105 -* https://github.com/donadigo/eddy/issues/107 - -### The Pop!_Shop should not default to the Visual Studio Code Flatpak - -Flatpaks are great most for applications, except for development tools. -Using the Visual Studio Code Flatpak makes configuring development tools harder. - -### Visual Studio Code should not prompt users to download a .tar.gz archive of Java without further instructions - -On Linux distributions where the distribution package manager provides a reasonably recent version of Java, users can install Java through the package manager more easily. diff --git a/programming/on-llms.md b/programming/on-llms.md deleted file mode 100644 index 744c2a38..00000000 --- a/programming/on-llms.md +++ /dev/null @@ -1,123 +0,0 @@ -# On LLMs - -This document is mostly an attempt to sort my thoughts on LLMs. -My intention is to flesh it out over time. - -I recently saw a survey about LLMs that had four options: they are great, they are hype, they are harmful, they are fake. -My thought was: it's all of the above. - -## LLMs are awesome - -I am absolutely amazed at LLMs. -By crunching an absurd amount of data, you get a magical box that can continue any dialogue, including dialogue that performs translations, writes code, and many others. -Although I suspect it's less technologically impressive, you can ask an LLM to draw a picture of anything and the LLM produces it! - -The components of LLMs are also magical. -When machine learning became a trend, many talented people found ways to turn things into vectors, which is amazing. -With LLMs, this trend continues. -Word embeddings can represent language as vectors, and we can operate with those vectors with semantic results. -This is an absolute breakthrough, with many profound implications. - -### LLMs seem great for accessibility - -Although I will elaborate later on my skepticism about some LLM results, I think that there are already valid applications of LLMs. - -Mostly, in accessibility. -LLMs are effective describing images and videos, can do text to speech and speech to text, and others. -Accessibility is a very important field and even if I have big objections to LLM use, I think LLMs are likely a net positive for accessibility. -Any criticism of LLM must take into consideration such applications, and if we want to avoid LLM use in these fields, we *must* provide equivalent alternatives. -(Some of my criticism below is ethical about LLM companies using content against their authors' wishes, but I think few authors would really object on not-for-profit accessibility usage of LLMs.) - -## LLMs are harmful - -### LLMs seem to require blatant disregard of intellectual property to be viable - -Napster (1999) was not the first high-profile case of massive copyright infringement, but since then, we have frequently seen large companies try to punish individuals to protect their intellectual property. - -With LLMs, courts have found out that companies training LLMs have used pirated material in a massive scale. - -I can see how for most people this is hugely unfair. - -Intellectual property is a complex topic, but morally it does not seem defensible to me that someone can be in trouble for pirating a TV show while a large company can try to make a lot of money through pirating massively. - -(As I mentioned, if this is not done for profit, to me this whole problem disappears.) - -I have mixed feelings with regard to copyright law, but ultimately I think that authors deserve to have rights over the use of their work. -For example, I think it is fair that a song composer might forbid an organization such as a political party from using their songs. -Art ultimately benefits us, and I think some degree of copyright protection helps incentivize creators. - -Therefore, given that LLM companies have widely admitted to using copyrighted material without permission, for-profit LLM companies must demonstrate that they have the right to use materials for training, and authors should have an easy way to prevent their works from being used, and to receive fair compensation if they want. - -My position does not hinge on how transformative the LLM output is. -It hinges on what authors want from their work; I am pretty certain that most authors nowadays are OK with other authors taking inspiration from their work (because they were also certainly inspired by other authors), but they are not OK with a large company reselling their work without compensation, or being used to replace authors with a machine. - -For other areas, such as writing programs using LLMs, I have similar objections. - -LLMs are effective as long as they have sufficient training material. -In my opinion, if you think the LLM would not be effective if all content from authors who would object to your specific use of an LLM was removed, then you should not use the LLM for that specific purpose. - -If you are using an LLM to create an illustration instead of commissioning an illustration, I think you should not do it because I don't think the LLM would be effective if all authors who objected to this use could remove their work from the training set. - -(As mentioned, I think very few authors would object to an LLM describing images to a blind person without making an obscene profit.) - -#### LLMs might need to be forced to publish all their output - -I read somewhere that LLM output should not be copyrightable. - -Making all LLM output public would solve a few issues: - -* Generated LLM content is now easy to identify by searching into the generated LLM output. -* IP laundering would result in public material that can be easily replicated and would make commercial benefit difficult. - -## LLMs are hype - -### LLMs might not be effective for writing code - -Hillel Wayne's [What we know we don't know](https://www.hillelwayne.com/talks/ese/ddd/) says that there are very few programming practices that are provably effective in increasing programming productivity. - -Apparently, the most effective practice is code review, and its effectiveness is small compared to getting proper sleep, not being stressed, and working the right amount of hours. - -Many other practices, such as good variable naming, or static or dynamic typing, do not seem to have a big effect, although most of us believe that some specific practices make us much more productive. - -Many people claim that LLMs make them wildly more productive, and I even believe many of them truly believe so. - -However, whenever I pull the thread on such claims, I tend to find things that make me skeptic. - -I do not rule out that LLMs may increase productivity *today* for *some* tasks. -But I suspect they might *also* decrease productivity sometimes. -This lack of certainty of their effectiveness for coding *combined* with the rest of the problems I see with the use of LLMs drive me towards rejecting the use of LLMs for code. -They might help, but they might also hinder us, and I think it's better to be conservative with regard to their use. - -### LLMs cannot be good oracles - -My understanding of LLMs is that they are good at producing "things that look like X". - -If you ask them to create a picture of X, then they will produce something that looks like a picture of X. -This is likely what you want, so I think LLMs can be reasonably effective at this task. - -If you ask them to answer X, then they will produce some text that looks like the answer to X. -However, something that reads like the answer to a question is likely *not* an answer to the question. - -Although it is surprising that many times the LLM will produce a correct answer, I think ultimately LLMs are not a good way to answer questions because they do that accidentally. - -## LLMs are harmful - -Or rather, they point to serious problems. - -I see many programmers using LLMs to review their code or brainstorm. - -Although LLMs might be somewhat effective in those tasks, I am surprised because people do not rely on other people for this. - -To me, this points to people not having other people to collaborate with, or preferring to collaborate with a piece of software rather than someone else. - -Although this might not be unequivocally harmful, it worries me, and I think we should spend time and resources looking at this phenomenon. - -Personally, I think we should offer ourselves to others more, probably in a tit-for-tat fashion. - -## Other sources - -* [The Future of Software Development is Software Developers](https://codemanship.wordpress.com/2025/11/25/the-future-of-software-development-is-software-developers/) - > “But this time it’s different, Jason!” - > [...] - > And there’s another important distinction: in previous cycles, the technology worked reliably. - > We really could produce working software faster with VB or with Microsoft Access. diff --git a/programming/python/about_django.md b/programming/python/about_django.md deleted file mode 100644 index 37004abc..00000000 --- a/programming/python/about_django.md +++ /dev/null @@ -1,141 +0,0 @@ -# About Django - -Without more context, one of the technologies I recommend to everyone is Django. - -Django is a Python web framework with "batteries included". - -Web frameworks can provide more or less tools to write applications. -Typically, frameworks that provide fewer tools are more flexible and give developers more freedom to develop their applications in the best possible way. -Similarly, frameworks that provide more tools tend to guide you towards a specific way to writing applications, and typically, require more work if you want to deviate. - -In my opinion, many applications you might need to develop are very similar and have similar issues, and solving them ad-hoc for each project is a waste. -Therefore, I lean towards using frameworks that provide more batteries in most cases. - -(Certainly, there are projects that clearly need special approaches, or which deviate enough from any generic web framework.) - -In fact, most of the complaints described in this document are caused by Django having too few batteries, not too many! - -> [!TIP] -> [Django training wheels](https://github.com/alexpdp7/django-tws) is my project in alpha to address some of those shortcomings. - -## The Django admin - -Besides including more batteries than most other frameworks, and being in general a well-engineered framework in my opinion, Django includes the admin. - -The admin is a declarative way to build administrative sites where some users edit data stored in the application database. - -Many similar tools exist, but I have not found any other tool that can do so much. - -* The Django admin handles multi-table relationships very well, including picking foreign key targets and editing related table data. - For example, if a person entity has a "parent" related foreign key relationship, the Django admin provides a search functionality to pick a person's parent. - If the person entity has a list of children, the Django admin provides a way to add and edit children from the person form. - -* The Django admin has a simple, but useful for many scenarios permissions functionality, where editing certain entities is restricted to groups of users. - -The Django admin is frequently a big boost during the early development of database-backed applications, and sometimes it can provide value during a big part of the life of an application. - -Additionally, traditionally when working with frameworks without an equivalent facility, the friction of adding an interface to edit a piece of data can be large. -Developers pressed for time might opt to hardcode the data in the source code of the application, requiring code changes to modify certain behaviors of the application. -When the friction to add a user interface to edit such data is low, developers can configure the admin to let those users edit the data directly without going through the developers. - -## Django problems - -However, there are still many common issues for which batteries could exist, but that Django does not provide. - -### Django has no support or documentation about packaging Django projects - -Most Django projects have dependencies besides Django. -In order to develop and deploy Django applications, you likely must install other dependencies. -Django does not include documentation nor support to do this. - -Many different approaches and tools exist to manage Python project dependencies. -Understandably, endorsing one particular approach in Django could be controversial. -So Django leaves the choice of approach up to users. -Additionally, Django adds a few difficulties in Python project management, and users must figure out how to handle Django projects in their chosen approach. - -Several initiatives have tried to tackle this problem, notably: - -* https://github.com/radiac/nanodjango - -### Django settings are a partial solution - -Django provides settings to manage the configuration for a Django project. -You implement Django settings by writing a Python module. - -For example, the default Django template includes the following snippet to configure the database connection: - -``` -DATABASES = { - 'default': { - 'ENGINE': 'django.db.backends.sqlite3', - 'NAME': BASE_DIR / 'db.sqlite3', - } -} -``` - -Besides assigning a setting directly like in the preceding snippet, you can use Python code to assign settings. - -This allows you to tackle many common issues, such as setting up a different database connection for development and production, while keeping the production database credentials away from the source code repository. -There are many similar issues that you must tackle in nearly all projects. - -Several initiatives tackle some of those issues: - -* https://github.com/jazzband/dj-database-url provides a way to configure the database connection through an environment variable. - -### Django does not explain a development database workflow - -Django provides migrations to handle schema changes. -Migrations work well and are a valid solution to handle schema changes in production. - -However, while developing a Django application, you frequently need to make many temporary changes to the data definition until you find the right data definition. - -In my opinion, if you follow the Django documentation, then you might end up using migrations for those development schema changes. -This is awkward and problematic, and there are procedures to develop database changes that work better. - -I would like a command that recreates your database, applying unmigrated model changes. -This command could also have hooks to load sample data. -(Likely, Python code and not fixtures.) - -### Django only tackles database-based, server-side-rendered, non highly interactive web applications - -While certainly a huge amount of applications: - -* Revolve around data stored in a relational database -* Are better implemented as server-side rendering applications -* Do not require very complex or real-time interactions - -There are certainly many applications that do not fit this mold. - -In my opinion, focusing on database-based applications is a good decision. -Many Django features (like the admin) revolve around the database, and a framework oriented to other applications likely should be very different. - -However, more and more applications break the limits of server-side rendering, and while you can build such applications with Django, you need a lot of effort or finding additional libraries to use. - -For example: - -* [Django REST framework](https://www.django-rest-framework.org/) provides a layer to provide REST APIs on top of the Django ORM. -* Projects exist to add support for Django for front end frameworks such as [htmx](https://htmx.org/) or [Hotwire](https://hotwired.dev/). - These frameworks are an intermediate step between traditional server-side-rendered applications and JavaScript front ends, enabling most of the benefits of JavaScript front ends within the traditional server-side rendering approach. - -Additionally, providing an API is also useful beyond JavaScript front ends. -APIs are necessary for other purposes, such as implementing mobile apps to interact with your application, or just providing an API for programmatic access to your application. - -### Some common tasks should have more tutorial content - -The Django documentation is mostly for reference, covering all Django features, but with little content on how to use Django. -The items I list below likely are documented on books, websites, forums, etc. -If you know a good source for many of those, even if it is paid, feel free to let me know to add references. - -* Admin - * Restricting users to a subset of the instances of a model. - For example, users belong to organizations and users should only see instances of some model related to their organization. - The FAQ contains [How do I limit admin access so that objects can only be edited by the users who created them?](https://docs.djangoproject.com/en/5.1/faq/admin/#how-do-i-limit-admin-access-so-that-objects-can-only-be-edited-by-the-users-who-created-them), which is a very similar question and points to the features you need to use to achieve these goals. - (These requirements are often related to requiring [extending the existing User model](https://docs.djangoproject.com/en/5.1/topics/auth/customizing/#extending-the-existing-user-model).) - * Having a search UI for reference fields instead of dropdowns. - Many projects similar to the admin only offer dropdowns for reference fields. - This does not work when the referenced objects are more than a couple. - Django calls this [`raw_id_fields`](https://docs.djangoproject.com/en/5.1/ref/contrib/admin/#django.contrib.admin.ModelAdmin.raw_id_fields), and it is difficult to learn that this feature exists. - -## Further reading - -* [CRUD is an important unsolved problem](../crud_is_an_important_unsolved_problem.md) diff --git a/programming/python/creating_nice_python_cli_tools.md b/programming/python/creating_nice_python_cli_tools.md deleted file mode 100644 index b192da1a..00000000 --- a/programming/python/creating_nice_python_cli_tools.md +++ /dev/null @@ -1,40 +0,0 @@ -Following this advice can make your tools easy to install by others, pleasant to use, robust, cross-platform, and powerful. - -* Use [my suggestions for setting up Python projects](project_setup.md), particularly: - * Provide instructions for installing your tool using [pipx](https://github.com/pypa/pipx). - Using pipx, people can install and upgrade your script using a simple command that requires no administrative privileges (but it requires having Python and pipx installed). - * As you are using [uv](https://docs.astral.sh/uv/), following the indications above: - * Use [entry points](https://docs.astral.sh/uv/concepts/projects/config/#entry-points), so when installing your tool via pipx or other means, your scripts are added to the user's path. - * Dependencies you define will be installed automatically along with your application. - This reduces the effort users need to use your application if you need third-party libraries. - However, I would still advise to avoid unnecessary dependencies (for simple HTTP requests you can use the base library. If you do complex requests, then using a third-party library might be much simpler). - As you are using pipx, those dependencies will be installed to a isolated virtualenv, so they will not interfere with anything on your system. - * As your application is properly packaged, you can split your code into different Python files and use imports without issues. -* If your application requires secrets, such as credentials or others, consider using: - * The standard [getpass](https://docs.python.org/3/library/getpass.html) module. - This prompts for a string on the command line, hiding what the user types. - * The [keyring](https://pypi.org/project/keyring/) library. - This stores secrets using your operating system facilities. -* Use the [appdirs](https://pypi.org/project/appdirs/) library to obtain "user paths", such as the users directory for configuration, cache, or data. - appdirs knows the proper paths for Linux, macOS and Windows. - So for example, if your tool caches files and uses appdirs to find the cache directory, you might gain benefits such as cache files being excluded from backups. -* If your tool requires significant time to complete a process: - * Use the [tqdm](https://tqdm.github.io/) library to add a progress bar. - * But also consider using the standard [concurrent.futures](https://docs.python.org/3/library/concurrent.futures.html) module to add parallelism if you can. - The [map](https://docs.python.org/3/library/concurrent.futures.html#concurrent.futures.Executor.map) function is particularly easy to use. - Use it with a [ThreadPoolExecutor](https://docs.python.org/3/library/concurrent.futures.html#concurrent.futures.ThreadPoolExecutor) if the parallel tasks are IO-bound or invoke other programs, or with [ProcessPoolExecutor](https://docs.python.org/3/library/concurrent.futures.html#processpoolexecutor) if they perform significant CPU work in Python (to avoid the [GIL](https://wiki.python.org/moin/GlobalInterpreterLock)). - * Consider using the standard [logging](https://docs.python.org/3/library/logging.html) module with a format that uses a timestamp, so users can inspect how much time is spent in different parts of the program. - You can also use logging module to implement flags such as `--debug` and `--verbose`. -* Although fancier tools exist, the standard [argparse](https://docs.python.org/3/library/argparse.html) module is good enough for most argument parsing. - It has decent support for [sub-commands](https://docs.python.org/3/library/argparse.html#sub-commands), and the linked document describes a very nice pattern to define functions for sub-commands, under "One particularly effective way of handling sub-commands..." - Provide help text for non-obvious parameters. - argparse supports a lot of different argument types with a lot of functionality out of the box, such as enumerated options, integers, and file names. - The main reason for using a fancier argument parsing is that argparse does not have autocomplete support, but you can add [argcomplete](https://github.com/kislyuk/argcomplete) to an argparse program with minimal modifications to retrofit autocomplete. -* Remember that the standard [json](https://docs.python.org/3/library/json.html) module is built-in. - You can use it to add a mode to your tool that generates JSON output instead of human-readable output, for easy automation of your tool, maybe using [jq](https://stedolan.github.io/jq/) or [fx](https://github.com/antonmedv/fx). -* Use the standard [subprocess](https://docs.python.org/3/library/subprocess.html) module to execute other commands. - * Remember never to use `shell=True`, so among other things, your tool will work correctly with files using spaces in their names. - * Use `check=True` so if the subprocess fails, an exception will be raised. - This is likely the best default behavior, although the error is a bit ugly, this normally prevents ugly problems and it's a safe option. - -You can find examples for many of those techniques in my [repos](https://github.com/alexpdp7?tab=repositories&q=&type=&language=python&sort=). diff --git a/programming/python/dependency_handling.md b/programming/python/dependency_handling.md deleted file mode 100644 index 8acf0cee..00000000 --- a/programming/python/dependency_handling.md +++ /dev/null @@ -1,111 +0,0 @@ -# Some brief notes about Python dependency management - -This article is mostly written for people who have already used Setuptools and have faced issues derived from its "limitations". -Specifically, if you have seen files named `requirements.txt` and have wondered how they work, what problem do they solve, and if they are something you should investigate, I hope you find this article interesting. - -If you are starting to write Python software and you are looking at an introductory text about distributing your software and using dependencies, I would recommend you to skip directly to using the "new generation" Python packaging tools. -This way, you can avoid most of the complexities in this post. -You can also check out the [Python Packaging User Guide](https://packaging.python.org/en/latest/) and [my own prescriptive project setup recommendations](project_setup.md). - -Most programs can use third-party libraries to implement parts of their functionality without implementing everything from scratch. - -pip is the recommended package installer for Python. -Python installers include pip, although pip is a component that can be installed separately from Python. -Some Linux distributions separate pip from the main Python package (for example, Debian has a `python3` package and a `python3-pip` package), but a Python install without `pip` is not really fully functional for many purposes. - -pip fetches Python packages from diverse sources and adds them to a Python installation. -Python packages can specify other packages as dependencies, so when pip installs a package, it also installs the required dependency chain. - -The traditional mechanism for packages to specify dependencies is Setuptools and other closely related projects. - -## About Setuptools - -Setuptools is a build and distribution system based on the distutils module that was part of the base Python library. - -Package metadata in Setuptools can be defined in many different ways, such as a `setup.py` file, a `setup.cfg` file, or a `pyproject.toml` file. -In these files, you list the dependencies for your package, specifying the name of the package and constraints. - -Constraints define which version of a dependency you want to use. -The constraint does not be an exact version, it can also be a range of versions, or a constraint such as "lower than version n". - -(Constraints additionally can specify other restrictions, such as requiring different versions for different Python versions, and other interesting possibilities.) - -In my opinion, although you can package applications and libraries properly using Setuptools, doing it correctly requires much knowledge, effort, and is error-prone. - -## Version locking and `requirements.txt` - -There is a dependency-management approach that can be very effective in many cases. - -This approach involves differentiating between "applications" and "libraries". - -Libraries are Python packages meant to be used as a dependency by other Python code. -Applications are Python code that may use other libraries as dependencies, but which no other Python code depends on. - -### Specifying dependencies for libraries - -Libraries specify coarse but safe dependency requirements. - -Suppose that we are developing the foo library. -The foo library depends on the bar library. -The bar library uses a versioning scheme similar to semantic versioning. -When we develop the foo library, we use version 1.2.3 of the bar library. - -Then, we specify that the foo library depends on the bar library, with a version constraint like `>=1.2.3, <1.3`. -This version constraint lets the library to be used with the 1.2.4 version, which is likely compatible with the code in the foo library, and even introduce valuable bug fixes. -However, the 1.3.0 version of the bar library would not be a valid dependency. -This is probably a good idea; the 1.3.0 may contain changes that the foo code is incompatible with. -(When we later create new versions of the foo library, we may want to consider depending on newer versions of the bar library, and possibly update the code so it continues working correctly.) - -This helps reduce conflicts. -As libraries specify coarse dependencies, the chances of two libraries having incompatible requirements is lower. -However, specifying coarse dependencies probably requires more testing to ensure that if different dependency versions are installed, the library works correctly. - -### Specifying dependencies for applications - -Applications specify exact dependency requirements. - -While libraries are not usually run on their own, applications are executed directly by end users. -If a library does not work well, then you can temporarily go back to an older version or apply other fixes. -But if an application does not work correctly, you have worse problems. - -If you specify exact dependency versions for an application, users of the application will always use a single combination of dependencies, which makes making things robust easy. - -A popular approach is for applications to specify Setuptools requirements with coarse versioning (just like libraries do), but to provide a list of the specific versions used for development and deployment. -To create this list of dependencies, you can install your application using pip or some other mechanism, then extract a list of the dependency versions that were installed and store it in a file. -For example, you can do this by executing: - -``` -$ pip install . # executed from the root of the application source code -$ pip freeze >requirements.txt -``` - -Later on, if you install the application using the following command: - -``` -$ pip install -r requirements.txt -``` - -Then you will always install the same set of dependencies, preventing issues by updated dependencies. - -Note: pip and other package installers do *not* use `requirements.txt` or any other similar file outside the `setup.cfg` file and the other files defined in Setuptools. -If you do not install your application explicitly using `pip install -r requirements.txt`, you will probably install a different set of dependencies. - -## Beyond version locking - -Following the approach above can be enough to use dependencies correctly. - -However, maintaining the Setuptools version dependencies and `requirements.txt` is straightforward, but tedious. -Also, this approach of dependency management is not obvious, and may not be easy to get right completely. - -For these reasons, several projects have appeared that implement approaches similar to the one described above, but more automatic and prescriptive. -These projects often manage automatically a file equivalent to `requirements.txt`, while the developer only specifies coarse dependencies for applications. - -Some of these tools are listed by [a page about relevant projects about packaging](https://packaging.python.org/en/latest/key_projects/) maintained by the [Python Packaging Authority](https://www.pypa.io/). -Look for tools about managing dependencies and packaging. - -Thanks to some improvements in the Python ecosystem, pip can nowadays install dependencies using many different packaging tools correctly. - -These projects can also offer some other improvements, so I would encourage Python developers to investigate them and try them out. - -However, also note that following a correct approach, Setuptools and manual version locking are perfectly valid ways to manage Python code dependencies. -Also, there are projects such as [pip-tools](https://github.com/jazzband/pip-tools) that complement Setuptools, addressing many of the issues described here, without requiring entirely new packaging tools. diff --git a/programming/python/project_setup.md b/programming/python/project_setup.md deleted file mode 100644 index a5f0c789..00000000 --- a/programming/python/project_setup.md +++ /dev/null @@ -1,114 +0,0 @@ -There is a significant amount of Python project tooling. This document collects my personal recommendations on how to set up a Python project. - -It is not meant to reflect the best or most common practices, just my personal taste. - -# Use pipx - -Pipx is a tool that installs Python packages to your user environment. It creates an isolated environment for every tool, so if you install multiple packages they won't have version conflicts. It also takes care of adding a module's entrypoints to your user path. - -uv can do very much the same and additionally uv can install most Python versions. -However, at the time of writing this, pipx is available as a package in many Linux distributions, while uv is not. - -If your project can be packaged so that it works with pipx, then many Linux users will be able to install it with pipx after installing pipx with their package manager. - -uv can be more convenient for software that requires specific versions of Python that are not available in Linux distributions, but in general cannot be installed with Linux package managers. - -# Use uv - -When using third-party dependencies in your Python code, it is highly interesting to avoid installing any project-specific dependency outside the project. - -To achieve that, traditionally virtualenvs are used; those are miniature Python installations where you can install any library you want. Virtualenvs need to be explicitly activated to be used, so it is easy to have a virtualenv for each Python project you are working on. - -uv is a tool that leverages virtualenvs to manage a project's dependencies, managing virtualenvs automatically. -uv can also manage Python distributions, downloading automatically Python versions other than the existing ones on your system. - -There are many similar tools such as pipenv and there are many multiple ways to specify a project's dependencies (`setup.py`, `requirements.txt`, etc.); uv provides a convenient way to do everything. - -Consider reading [some brief notes about Python dependency management](dependency_handling.md). - -# Test your code - -Write the necessary amount of tests so you can make changes to your code with confidence. - -If you find yourself iterating over a piece of code slowly, try to isolate the code you are writing so it can be tested in isolation for faster iteration. - -## Use pytest for testing - -Python provides *two* testing frameworks in its standard library, but they have some limitations: - -* `unittest` is an xUnit-style testing framework which follows non-PEP-8 naming conventions (probably because it copied the Java's jUnit), so extra work needs to be done to make your test cases PEP-8 compliant -* `doctest` is a tool which allows you to run tests embedded in comments. For some code, it is great and helps you provide good, up-to-date documentation. However, a significant amount of code is awkward to test using `doctest`. - -Use `doctest` whenever you can, but outside that, use `pytest` to write PEP-8-compliant tests. - -Ensure that your test suite runs correctly by running `pytest` without any arguments. - -Use plain Python's `assert` statements to check assertions in your tests; `pytest` does some magic to provide nice error messages on failed assertions. - -## Gate your changes with testing - -Set up your version control so changes cannot be made to your main codeline without passing continuous integration tests (and possibly, code review). - -# Perform automated code formatting and static checking - -> [!NOTE] -> I have been using [ruff](https://github.com/astral-sh/ruff) recently. -> Not enough to recommend it unconditionally over flake8/black, but I am liking it so far. -> Consider testing it. -> It requires slightly less configuration and it comes with more lints. - -## Use Ruff - -Use Ruff to format and lint your code. - -# Version control - -## Use a minimal gitignore file - -See [use gitignore properly](../git/git_advice.md#use-gitignore-properly). - -## Keep your code together - -All the code you modify as part of the project should be kept in a single repository so you can make atomic changes. If you find yourself making changes across multiple repositories and having to coordinate them, consider merging those repositories. - -Use git submodules or similar mechanisms to refer to code you modify that must be kept external. - -Use [Josh](../git/combining_repos_with_josh_filter.md) to publish parts of the repository outside the main repository if needed. - -# Support multiple modern versions of Python - -Unless you have a specific requirement to support Python 2, don't. - -It is reasonable to support multiple versions of Python 3 from 3.4 onwards. Supporting the oldest versions might limit the features you can use (although features from more modern versions have been backported), so evaluate which operating systems and versions you need to support and try to support Python versions readily available for them (in Linux, by using mainline distro repos, for instance). - -Even if you are not running your code using the latest versions of Python, try to support all the newest available versions. - -Use continuous integration to run your tests in all supported versions of Python. - -# Use ipython and ipdb - -Add ipython and ipdb as development dependencies. - -# Versioning - -Unless you have a specific requirement to support multiple versions of your code or to distribute to a platform that *requires* versioning (such as pypi), do not explicitly version your code but allow implicit versioning (e.g. it should be possible to identify which Git commit deployed code comes from). - -# Documentation - -Provide a `README` containing: - -* The purpose of the code -* How to use the code -* How to develop the code - -If the `README` becomes unwieldly, separate usage instructions to `USAGE` and/or development instructions to `HACKING`. - -Provide docstrings detailing the external interface of Python modules. Provide internal comments in modules detailing implementation. - -If you are developing a library/framework, consider using Sphinx. Sphinx can create a documentation website for a Python project, taking advantage of docstrings. - -# Distribution - -If your code can be executed from a command line, consider documenting installation via `pipx`. - -If your code has dependencies that are not trivial to install (such as Pandas), consider publishing a Docker image or using dependencies that are simpler to install. Design your Docker images so rebuilding the image on most changes is fast. diff --git a/programming/python/python_modules_primer.md b/programming/python/python_modules_primer.md deleted file mode 100644 index d80b96d3..00000000 --- a/programming/python/python_modules_primer.md +++ /dev/null @@ -1,272 +0,0 @@ -# Python Modules Primer - -## Prerequisites - -These instructions assume a Linux environment. -A macOS environment is similar, but not identical. -A Windows environment is more different. - -## Previous knowledge - -### A refresher on the `PATH` variable - -If you execute the following command in your terminal: - -``` -$ echo hello -``` - -, the shell searches for the `echo` command in the directories listed in your `PATH` environment variable. -You can display your `PATH` variable by running: - -``` -$ echo $PATH -/home/user/.local/bin:/home/user/bin:/usr/share/Modules/bin:/usr/local/bin:/usr/local/sbin:/usr/bin:/usr/sbin -``` - -The contents of the `PATH` variable depend on your particular environment. - -If you run the following command: - -``` -$ which echo -/usr/bin/echo -``` - -The `which` command prints where the shell locates the `echo` command. - -### A refresher on shell scripts - -If you create a file named `foo.sh` with the following contents: - -``` -#!/bin/sh - -echo hello -``` - -You define a "shell script". -The first line indicates that this shell script is executed by using the `/bin/sh` command. -The rest of the file are commands to be executed by the shell command. -These commands behave as if you typed them into your terminal, so if you execute this script, the command `echo hello` will be executed, printing `hello`. - -If you try to run `foo.sh` like you run the `echo` command, by typing its name, it does not work: - -``` -$ foo.sh -bash: foo.sh: command not found... -``` - -, because the shell looks for the `foo.sh` in the directories listed in the `PATH` variable. -Unless you created the `foo.sh` file in a directory like `/usr/bin`, the shell will not find the `foo.sh` command. - -A solution to this problem is to specify the path to the `foo.sh` file, instead of relying on the `PATH` variable. -However, if you do this, you face a second problem. - -``` -$ ./foo.sh -bash: ./foo.sh: Permission denied -``` - -This happens because only files with the executable permission can be executed in this way. -To solve this, add the executable permission; then it works: - -``` -$ chmod +x foo.sh -$ ./foo.sh -hello -``` - -## The `import` statement in Python - -### Importing from the Python standard library - -Run the following commands by using the Python REPL: - -``` -$ python3 -Python 3.9.17 (main, Aug 9 2023, 00:00:00) -[GCC 11.4.1 20230605 (Red Hat 11.4.1-2)] on linux -Type "help", "copyright", "credits" or "license" for more information. ->>> import datetime ->>> datetime.datetime.now() -datetime.datetime(2023, 9, 11, 21, 53, 16, 331236) -``` - -`import` works in a similar way to running a command in the shell. -Python searches a number of directories looking for the `datetime` module. - -To see which directories are searched, run: - -``` -$ python3 ->>> import sys ->>> sys.path -['', '/usr/lib64/python39.zip', '/usr/lib64/python3.9', '/usr/lib64/python3.9/lib-dynload', '/home/alex/.local/lib/python3.9/site-packages', '/usr/lib64/python3.9/site-packages', '/usr/lib/python3.9/site-packages'] -``` - -`sys.path` is a list of the directories that the `import` command searches. -The contents of `sys.path` depend on your operating system and Python installation method. - -In my system, the `/usr/lib64/python3.9` directory contains the `datetime.py` module. - -``` -$ head /usr/lib64/python3.9/datetime.py -"""Concrete date/time and related types. - -See http://www.iana.org/time-zones/repository/tz-link.html for -time zone and DST data sources. -""" - -__all__ = ("date", "datetime", "time", "timedelta", "timezone", "tzinfo", - "MINYEAR", "MAXYEAR") -... -``` - -`/usr/lib64/python3.9` contains the modules in [the Python standard library](https://docs.python.org/3/library/). - -### Importing your Python files - -If you create a file with the `a.py` name: - -``` -def f(): - return 2 -``` - -, and another with the `b.py` name: - -``` -import a - -print(a.f()) -``` - -, then: - -``` -$ python b.py -2 -``` - - -This works, because `sys.path` contains `''`, which means "the current directory". - -(`sys.path` is very similar to the `PATH` variable. However, `sys.path` contains the current directory by default, whereas `PATH` does not.) - -When `import a` is executed, then Python searches the directories in `sys.path` for an `a.py` file; it is found when checking the `''` path. -When `import datetime` is executed, Python searches in the current directory (because `''` comes first in the path), doesn't find it, but then finds it in the following `/usr/lib64/python3.9` directory. -Python iterates over the `sys.path` directories, and loads the *first* matching file. - -## Installing libraries - -When writing Python software, sometimes it is enough with the modules included in the standard library. -However, frequently you want to use other libraries. -To use Python libraries, you must install them using the `pip` program. - -The `pip` program is not part of the `python3` package in some Linux distributions, and comes from the `python3-pip` package. - -The `pip` program can download libraries from https://pypi.org/ , the Python package index, and install them. -`pip` installs libraries to a "Python environment". - -Old versions of `pip` defaulted to installing libraries to the "system" Python environment. -In a Linux system, the system Python environment is located in a directory such as `/usr/lib64/python3.9`. -By default, normal Linux users cannot write to `/usr`, so installing a package would fail. - -Modern versions of `pip` detect that they cannot write to the "system" Python environment, and then redirect the install to the "user" Python environment. -The "user" Python environment is in a directory such as `~/.local/lib/python3.9`. - -You could use a command such as `sudo pip install` to grant `pip` the privileges required to write to `/usr`. -However, this can make a Linux system unusable. -Most Linux systems use software that uses the "system" Python environment. -Altering the "system" Python environment can break such software. -Do not run `sudo pip install` with root privileges unless you know why you need this. - -If you use a modern `pip` (or use the `--user` option), you can install libraries to the "user" Python environment. -However, this is problematic because a Python environment can only contain a single version of a Python library. -If you have two different Python programs that different versions of the same library, then these two programs cannot coexist in the "user" Python environment. - -In general, Python virtual environments are used to address this problem. - -## Creating Python virtual environments - -If you run: - -``` -$ python3 -m venv <some path> -``` - -This will create a directory with the path you specify, with the following contents: - -``` -<some path> -├── bin -│ ├── activate -│ ├── pip -│ ├── python -├── include -├── lib -│ └── python3.9 -``` - -The `python` and `pip` commands are copies of the same commands from the "system" Python environment. - -But these commands work differently from the "system" Python environment commands: - -``` -$ <some path>/bin/python ->>> import sys ->>> sys.path -['', '/usr/lib64/python39.zip', '/usr/lib64/python3.9', '/usr/lib64/python3.9/lib-dynload', '<some path>/lib64/python3.9/site-packages', '<some path>/lib/python3.9/site-packages'] -``` - -`sys.path` uses the `lib` directories in the virtual environment. - -When you use the `pip` program from the virtual environment, it installs the libraries to the virtual environment. - -You can create as many virtual environments as you need, and you can install different versions of libraries to each virtual environment. - -## Activating Python environments - -You can run the `python` and `pip` commands by specifying the full path, like we did when executing the `foo.sh` command earlier. - -By default, if you run `python`, the shell will invoke the `python` command from the "system" Python environment because it is in a directory included in the `PATH` variable. -If you specify the full path, you override this. - -To save typing, the `bin` directory of a virtual environment contains an `activate` file. -The `activate` file is a "special" shell script that must be invoked in one of the following two ways: - -``` -$ source <some path>/bin/activate -``` - -``` -$ . <some path>/bin/activate -``` - -`source` and `.` are synonyms. -They are special shell commands that are needed for the `activate` command to work correctly. - -`activate` alters your path, so that the `bin` directory in your virtual environment comes first in your path. - -``` -$ echo $PATH -/home/user/.local/bin:/home/user/bin:/usr/share/Modules/bin:/usr/local/bin:/usr/local/sbin:/usr/bin:/usr/sbin -$ . <some path>/bin/activate -(some path) $ echo $PATH -<some path>/bin:/home/user/.local/bin:/home/user/bin:/usr/share/Modules/bin:/usr/local/bin:/usr/local/sbin:/usr/bin:/usr/sbin -``` - -, and thus if you run `python`, `<some path>/bin/python` will be executed instead of `/usr/bin/python`. - -Besides changing your prompt to indicate the virtual environment is activated, `activate` only alters your `PATH`. -`activate` is not mandatory to use a virtual environment. -For example, when running the Python command, if you specify the path of the Python executable in a virtual environment, the command will execute as if the virtual environment had been activated. -Tools such as `poetry` have commands such as `poetry run` that can run commands inside a virtual environment without activating it. -Activation can save time, but it is also more error-prone than more explicit means of using virtual environments. - -## Further reading - -* [Some brief notes about Python dependency management](dependency_handling.md) continues this explanation, introducing the need for packaging tools. -* [Installing Python Modules](https://docs.python.org/3/installing/index.html), from the official Python documentation, describes the `pip` program in more depth. -* [`venv` - Creation of virtual environments](https://docs.python.org/3/library/venv.html), from the official Python documentation, describes virtual environments in more depth. diff --git a/programming/python/scraping_with_selenium_on_docker.md b/programming/python/scraping_with_selenium_on_docker.md deleted file mode 100644 index 61ba1c12..00000000 --- a/programming/python/scraping_with_selenium_on_docker.md +++ /dev/null @@ -1,7 +0,0 @@ -Don't use Selenium, use [Playwright](https://playwright.dev/python/): - -* Playwright automatically sets up headless browsers. -* Provides convenient abstractions for locating elements in a page (mostly no XPath required. It can match "intelligently" using text). -* Has a handy UI tool that records your actions in a browser and writes equivalent *readable* Playwright code. - -Further reading: https://new.pythonforengineers.com/blog/web-automation-dont-use-selenium-use-playwright/ diff --git a/programming/the-content-web-manifesto/NOTES.org b/programming/the-content-web-manifesto/NOTES.org deleted file mode 100644 index b56c4bf1..00000000 --- a/programming/the-content-web-manifesto/NOTES.org +++ /dev/null @@ -1,30 +0,0 @@ -* [[README.md][README]] - -- https://www.fixbrowser.org/ - -** Document how terminal browsers can invoke a full browser to execute JavaScript - -See [[https://www.gnu.org/software/emacs/manual/html_node/eww/Advanced.html]], w3m has similar stuff. - -Also: - -- https://github.com/abhinavsingh/proxy.py Extensible Python proxy -- https://github.com/TempoWorks/txtdot -- https://github.com/4383/photonos -- https://sr.ht/%7Ebptato/chawan/ -- https://offpunk.net/ - -Browsers as a platform to manage content: - -- Just view the content as HTML with user-defined styling -- Archive all that we see so that we can locate content we have read easily, share with others, etc. -- RSS/Gemfeed/content subscription - -** Annotate URLs with another URLs - -- For example, add transcriptions to comic strips that do not have them -- The server pushes serialized bloom filters of annotated URLs (or entire annotation sets?) so that clients do not have to leak what they are browsing. -- Maybe https://dokie.li/ -- Alternative approach with Violentmonkey for accessibility purposes: [[https://github.com/alexpdp7/aelevenymonkey]]. - -** NoScript configuration merge diff --git a/programming/the-content-web-manifesto/README.md b/programming/the-content-web-manifesto/README.md deleted file mode 100644 index d05256b9..00000000 --- a/programming/the-content-web-manifesto/README.md +++ /dev/null @@ -1,52 +0,0 @@ -# The content web manifesto - -These are my recommendations for creating "content" websites. -In a content website visitors mostly read content. -Some example content websites are Wikipedia, news websites, and blogs. - -Also see [further notes](NOTES.org). - -## General guidelines - -### Test your website with a terminal browser without JavaScript like w3m, lynx, or elinks - -If your website is usable with one of those browsers, then: - -* Your website does not require JavaScript to load. - This automatically addresses most annoyances with content websites. - Websites that do not require JavaScript tend to require less resources, making them faster and lighter. - -* Your website does not rely on non-text content. - Text content is uniquely flexible, it is frequently the most amenable media to being processed by the following systems and processes: - - * Text-to-speech systems - * Translation (both human and automatic) - * Edition (making changes to text content) - * Quoting/embedding (readers can copy parts of your text to cite or promote your content) - - Images, audio, video or other interactive media might be required to convey the message of your content. - Therefore, the content web manifesto does not forbid their use. - However, non-text content should always be accompanied by at least a text description of the content, and ideally, an alternate text version of the content. - -* Your website will work with user styling. - Providing a visual style via CSS and others is fine, but users should be able to read your content with *their* choice of font, text size, color, and others. - This is important for accessibility, but also for everyone's comfort. - -And more importantly, this weakens browser monopolies controlling the web. -Not even massive companies like Microsoft dare to maintain a browser engine, leaving the web subject to the power of the very few browser vendors in existence. -But if your web content can be read under a terminal browser without Javascript, then your content is automatically accessible by a massive amount of browsers, including very simple ones. - -(Alternatively, use [the Gemini protocol](https://geminiprotocol.net/).) - -### Provide granular URLs - -When providing a significant amount of content, make sure readers can link to specific content of interest. - -This can be achieved by: - -* Splitting your content in different pages -* Providing HTML headers with anchors - -### Date content - -Always make initial publication and edition dates available. diff --git a/programming/we_cant_code_good_so_take_llm_hype_with_a_grain_of_salt.md b/programming/we_cant_code_good_so_take_llm_hype_with_a_grain_of_salt.md deleted file mode 100644 index 675f8a2e..00000000 --- a/programming/we_cant_code_good_so_take_llm_hype_with_a_grain_of_salt.md +++ /dev/null @@ -1,32 +0,0 @@ -# We can't code good (so take LLM hype with a grain of salt) - -Software engineering is a young field compared to most other fields. - -All studies about software engineering projects I have seen say that most projects are late, over budget, and underdeliver. - -(A great article did some statistics to estimate the percentage of software engineers who have never seen worked on a successful project. -The result was discouraging. -I have never found this article again, please contact me if you know it.) - -Consider CRUD applications, which most software engineers consider to be the lowest kind of work. -A massive amount of CRUD applications have been developed, and [I believe we are not effective at churning them out](crud_is_an_important_unsolved_problem.md). - -[What we know we don't know: empirical software engineering](https://www.hillelwayne.com/talks/ese/ddd/) is an interesting talk by Hillel Wayne that elaborates on this topic. - -The talk claims that researchers who try to prove that any engineering practice has a significant effect on delivery mostly fail. -Apparently, code review seems to be one of the few practices that has been proven to be effective but its effectivity is small compared to sleeping well, not being stressed, or working the right amount of hours. - -This includes practices that most of us believe have significant effects, such as using dynamic or static typing, abbreviating or not identifiers, and many others. - -We all believe that some practices are effective, yet this does not seem to be measurable. - -I think experience, good team work, smarts, and motivation help. -(And I believe there are certainly things that *hinder* software engineering!) -We have been able to deliver good works of engineering, and some things seem to work, but we can't code good yet. - -So *any* claim of *any* practice being effective should be taken with a dose of healthy skepticism. -I do not even know how to measure software engineering effectivity; I can use my experience to compare and try to infer conclusions, but I know my conclusions can be wrong. - -I am sure that at some point our discipline will be as mature as other disciplines, but think how much effort it took other disciplines to get there. -But until then, we should be humble and not make bold claims, and we should question the incentives of those who make bold claims. -Perhaps some of those bold claims turn out to be true, but the longer that you have been in this field, the more bold claims you will have seen fail. diff --git a/programming/when_to_code.md b/programming/when_to_code.md deleted file mode 100644 index c0bead9e..00000000 --- a/programming/when_to_code.md +++ /dev/null @@ -1,18 +0,0 @@ -# When to code - -Our job as programmers is to solve real-world problems in a satisfactory manner. -As we are coders, our first instinct is to jump in and write some code. -This is often a mistake, consider: - -* Coding is very expensive. -It takes a lot of time, not just coding, but gathering requirements from the business people, deploying it to a production environment, doing end-user testing, maintaining it... -* There is a massive amount of software out there- chances are, someone had the need before and paid someone to do it. -If it is a common problem, there is a chance that there is something that will fit exactly your needs (and if it doesn't fit exactly, you might need to rethink your requirements- someone went over the full process and probably you have not- they might have figured out something you have not). -Perhaps it's got a price tag on it, and maybe you think it's excessive, but if they are commercializing it and they have a few customers, consider that the cost of development will be divided among all customers- and you might not have this advantage. -* Even if there is no specific off-the-shelf software which solves your needs, some generic software such as an spreadsheet or a content management system, or some specific features such as a word processor's mail merge functionality might come close enough to your needs. -It is likely that at some point you will outgrow it (and software such as spreadsheets are specially amenable to being twisted and stretched terribly beyond their intended use case, resulting in terrible monsters which you should be wary of), but starting with them might help you frame the problem better than starting with code. - -Still, there are lots of uncharted territories- there are needs for which no decent off-the-shelf software exists, or you might have already examined all possibilities and they truly don't fit you. -Or maybe you want to create a competing product, or doing something truly innovative. -Or (this is a common scenario) you are not able to convince someone to use an off-the-shelf solution. -In that case, you will have to code. diff --git a/scripts/paperwm b/scripts/paperwm new file mode 100755 index 00000000..c05285b8 --- /dev/null +++ b/scripts/paperwm @@ -0,0 +1,18 @@ +#!/bin/sh + +set -uex + +test -f ~/.cache/paperwmpaperwm.github.com.v145.shell-extension.zip || { + wget https://extensions.gnome.org/extension-data/paperwmpaperwm.github.com.v145.shell-extension.zip -O ~/.cache/paperwmpaperwm.github.com.v145.shell-extension.zip +} + +test -d ~/.local/share/gnome-shell/extensions/paperwm@paperwm.github.com || { + mkdir -p ~/.local/share/gnome-shell/extensions/paperwm@paperwm.github.com + cd ~/.local/share/gnome-shell/extensions/paperwm@paperwm.github.com + unzip ~/.cache/paperwmpaperwm.github.com.v145.shell-extension.zip +} + +dconf load / <<EOF +[org/gnome/shell] +enabled-extensions=['paperwm@paperwm.github.com'] +EOF |