diff options
Diffstat (limited to 'programming')
| -rw-r--r-- | programming/crud_is_an_important_unsolved_problem.md | 92 | ||||
| -rw-r--r-- | programming/python/about_django.md | 141 |
2 files changed, 0 insertions, 233 deletions
diff --git a/programming/crud_is_an_important_unsolved_problem.md b/programming/crud_is_an_important_unsolved_problem.md deleted file mode 100644 index 88497f7e..00000000 --- a/programming/crud_is_an_important_unsolved_problem.md +++ /dev/null @@ -1,92 +0,0 @@ -# CRUD is an important unsolved problem - -The CRUD (initials of create, replace, update, and delete) term is used to describe the implementation of applications that provide a simple interface to a database. - -If you have used a computer for work, then you likely have used a system that allows to manipulate records of customers, orders, products, or any other information related to the business. - -Although programmers have been written a huge amount of CRUD systems for decades, my perception is that the costs of implementing CRUD systems is a major problem with important consequences. - -There are two major approaches to implementing CRUD systems: - -* Traditional programming: combining a relational database and most existing programming languages enables programmers to create CRUD systems. - -* "No code" (or "low code"): many products and services enable non-programmers to describe their data structure and the user interface requiring less technical knowledge than the traditional programming approach. - -## About implementing CRUD systems with traditional programming - -The Python Django web framework coupled with a relational database requires writing the least code of all the platforms I have used to write CRUD systems. - -Most of what you need to do when using Django is to describe what you need, instead of implementing the mechanical parts of a CRUD system. - -Out of the box, Django provides: - -* List and detail views, including nested views. - Many systems provide "flat" details view where you can edit a record, but not associated records. - For example, they provide a detail view for customers where you can edit the customer name and other information, but any "multiple" information, such as multiple addresses or phone numbers, must be edited in a different view. - This is frequently a huge issue, and it can require writing a significant amount of code in other systems. - With Django, you can implement this by describing the associated data. - -* Multi user authentication and role-based authentication. - With Django and without programming any code, administrators can create groups, assign users to groups, and limit the kinds of records that each group can view or edit. - -* Primitive change tracking. - Out of the box, changes to records are tracked automatically and can be consulted. - -For most CRUD implementations, alternative platforms require significantly more effort to implement those features. - -Additionally, the entire stack is open source software that does not require paying licenses. - -(Surprisingly, in the past there existed even more sophisticated CRUD platforms. - But sadly, most have disappeared.) - -## About implementing CRUD systems with no code - -A huge amount of systems provide similar functionality, in an even more friendly manner. - -They typically provide a user interface where you can create tables, add columns, and describe the user interface without programming. - -Some of those systems offer features comparable or superior to Django. - -However, because those systems focus on no code usage, frequently you hit roadblocks when using them. - -When you need a feature that they do not provide, it is either impossible to do it, or it requires programming in an unfriendly environment. - -Programming CRUD features can be complex. -While traditional programming tools have evolved providing many features such as automated testing and advanced code revision control systems (rolling back bad changes and others), no code CRUD platforms do not reuse the wealth of programming tools that have been developed for traditional programming. - -Non-developers frequently face huge challenges going beyond the basics of what the tool provides, and developers struggle and suffer by working in environments that are more limiting compared to others. - -## The consequences of the high cost of development of CRUD systems - -In these conditions, most CRUD systems are expensive and do not work well. - -Organizations often resort to systems such as spreadsheets that can be productive, but have severe reliability concerns. - -No code CRUD systems often have significant costs and lock in their customers, because migrating costs can be astronomical. - -CRUD systems implemented with traditional programming often are costly to maintain and extend. - -In most cases, organizations cannot justify the costs of tailoring the CRUD system entirely to their needs, so they suffer from using CRUD systems that do not meet their needs. - -## Possible approaches - -### Improving existing traditional programming CRUD platforms - -I believe systems such as Django can still see many improvements. -Likely, both the amount of technical knowledge to use these systems and the amount of effort to design CRUD systems can be reduced significantly. - -### Providing systems to transition from no code approaches to traditional programming - -No code approaches are wonderful, because giving end users the ability to describe what they need enables them to experiment and become productive very quickly. - -However, no code platforms cannot provide all features needed, and in many cases, end users will struggle past a certain point. - -Providing a way to migrate to a traditional programming approach would enable breaking this barrier and scaling systems more effectively. - -(Some no code platforms have APIs. - With them, programmers can write code to extend the no code CRUD systems using traditional programming approaches. - However, implementing functionalities through APIs has limitations and specific problems.) - -## Further reading - -* [About Django](python/about_django.md). diff --git a/programming/python/about_django.md b/programming/python/about_django.md deleted file mode 100644 index 37004abc..00000000 --- a/programming/python/about_django.md +++ /dev/null @@ -1,141 +0,0 @@ -# About Django - -Without more context, one of the technologies I recommend to everyone is Django. - -Django is a Python web framework with "batteries included". - -Web frameworks can provide more or less tools to write applications. -Typically, frameworks that provide fewer tools are more flexible and give developers more freedom to develop their applications in the best possible way. -Similarly, frameworks that provide more tools tend to guide you towards a specific way to writing applications, and typically, require more work if you want to deviate. - -In my opinion, many applications you might need to develop are very similar and have similar issues, and solving them ad-hoc for each project is a waste. -Therefore, I lean towards using frameworks that provide more batteries in most cases. - -(Certainly, there are projects that clearly need special approaches, or which deviate enough from any generic web framework.) - -In fact, most of the complaints described in this document are caused by Django having too few batteries, not too many! - -> [!TIP] -> [Django training wheels](https://github.com/alexpdp7/django-tws) is my project in alpha to address some of those shortcomings. - -## The Django admin - -Besides including more batteries than most other frameworks, and being in general a well-engineered framework in my opinion, Django includes the admin. - -The admin is a declarative way to build administrative sites where some users edit data stored in the application database. - -Many similar tools exist, but I have not found any other tool that can do so much. - -* The Django admin handles multi-table relationships very well, including picking foreign key targets and editing related table data. - For example, if a person entity has a "parent" related foreign key relationship, the Django admin provides a search functionality to pick a person's parent. - If the person entity has a list of children, the Django admin provides a way to add and edit children from the person form. - -* The Django admin has a simple, but useful for many scenarios permissions functionality, where editing certain entities is restricted to groups of users. - -The Django admin is frequently a big boost during the early development of database-backed applications, and sometimes it can provide value during a big part of the life of an application. - -Additionally, traditionally when working with frameworks without an equivalent facility, the friction of adding an interface to edit a piece of data can be large. -Developers pressed for time might opt to hardcode the data in the source code of the application, requiring code changes to modify certain behaviors of the application. -When the friction to add a user interface to edit such data is low, developers can configure the admin to let those users edit the data directly without going through the developers. - -## Django problems - -However, there are still many common issues for which batteries could exist, but that Django does not provide. - -### Django has no support or documentation about packaging Django projects - -Most Django projects have dependencies besides Django. -In order to develop and deploy Django applications, you likely must install other dependencies. -Django does not include documentation nor support to do this. - -Many different approaches and tools exist to manage Python project dependencies. -Understandably, endorsing one particular approach in Django could be controversial. -So Django leaves the choice of approach up to users. -Additionally, Django adds a few difficulties in Python project management, and users must figure out how to handle Django projects in their chosen approach. - -Several initiatives have tried to tackle this problem, notably: - -* https://github.com/radiac/nanodjango - -### Django settings are a partial solution - -Django provides settings to manage the configuration for a Django project. -You implement Django settings by writing a Python module. - -For example, the default Django template includes the following snippet to configure the database connection: - -``` -DATABASES = { - 'default': { - 'ENGINE': 'django.db.backends.sqlite3', - 'NAME': BASE_DIR / 'db.sqlite3', - } -} -``` - -Besides assigning a setting directly like in the preceding snippet, you can use Python code to assign settings. - -This allows you to tackle many common issues, such as setting up a different database connection for development and production, while keeping the production database credentials away from the source code repository. -There are many similar issues that you must tackle in nearly all projects. - -Several initiatives tackle some of those issues: - -* https://github.com/jazzband/dj-database-url provides a way to configure the database connection through an environment variable. - -### Django does not explain a development database workflow - -Django provides migrations to handle schema changes. -Migrations work well and are a valid solution to handle schema changes in production. - -However, while developing a Django application, you frequently need to make many temporary changes to the data definition until you find the right data definition. - -In my opinion, if you follow the Django documentation, then you might end up using migrations for those development schema changes. -This is awkward and problematic, and there are procedures to develop database changes that work better. - -I would like a command that recreates your database, applying unmigrated model changes. -This command could also have hooks to load sample data. -(Likely, Python code and not fixtures.) - -### Django only tackles database-based, server-side-rendered, non highly interactive web applications - -While certainly a huge amount of applications: - -* Revolve around data stored in a relational database -* Are better implemented as server-side rendering applications -* Do not require very complex or real-time interactions - -There are certainly many applications that do not fit this mold. - -In my opinion, focusing on database-based applications is a good decision. -Many Django features (like the admin) revolve around the database, and a framework oriented to other applications likely should be very different. - -However, more and more applications break the limits of server-side rendering, and while you can build such applications with Django, you need a lot of effort or finding additional libraries to use. - -For example: - -* [Django REST framework](https://www.django-rest-framework.org/) provides a layer to provide REST APIs on top of the Django ORM. -* Projects exist to add support for Django for front end frameworks such as [htmx](https://htmx.org/) or [Hotwire](https://hotwired.dev/). - These frameworks are an intermediate step between traditional server-side-rendered applications and JavaScript front ends, enabling most of the benefits of JavaScript front ends within the traditional server-side rendering approach. - -Additionally, providing an API is also useful beyond JavaScript front ends. -APIs are necessary for other purposes, such as implementing mobile apps to interact with your application, or just providing an API for programmatic access to your application. - -### Some common tasks should have more tutorial content - -The Django documentation is mostly for reference, covering all Django features, but with little content on how to use Django. -The items I list below likely are documented on books, websites, forums, etc. -If you know a good source for many of those, even if it is paid, feel free to let me know to add references. - -* Admin - * Restricting users to a subset of the instances of a model. - For example, users belong to organizations and users should only see instances of some model related to their organization. - The FAQ contains [How do I limit admin access so that objects can only be edited by the users who created them?](https://docs.djangoproject.com/en/5.1/faq/admin/#how-do-i-limit-admin-access-so-that-objects-can-only-be-edited-by-the-users-who-created-them), which is a very similar question and points to the features you need to use to achieve these goals. - (These requirements are often related to requiring [extending the existing User model](https://docs.djangoproject.com/en/5.1/topics/auth/customizing/#extending-the-existing-user-model).) - * Having a search UI for reference fields instead of dropdowns. - Many projects similar to the admin only offer dropdowns for reference fields. - This does not work when the referenced objects are more than a couple. - Django calls this [`raw_id_fields`](https://docs.djangoproject.com/en/5.1/ref/contrib/admin/#django.contrib.admin.ModelAdmin.raw_id_fields), and it is difficult to learn that this feature exists. - -## Further reading - -* [CRUD is an important unsolved problem](../crud_is_an_important_unsolved_problem.md) |