[Feature]: Implement Async Views and Database Operations in Django for Enhanced Performanc

[brylie]

Feature Description

Port existing views and database operations in the project to utilize Python/Django's asynchronous features. This enhancement involves refactoring current synchronous views and database interactions to asynchronous ones using the latest async capabilities provided by Django. The objective is to improve the overall performance and scalability of the application by enabling non-blocking database operations and handling concurrent requests more efficiently.

Motivation

As the application grows in complexity and user base, performance and scalability become increasingly important. Traditional synchronous processing can lead to inefficiencies, especially under heavy load. By adopting Django's async features, we can handle more requests simultaneously, reduce response times, and improve user experience, particularly in data-intensive operations. This shift is also in line with modern web development practices, ensuring our application remains robust and future-proof.

Possible Implementation

• Refactor views to asynchronous using Django's async def where appropriate.
• Utilize asyncio along with Django's support for async ORM queries for database operations.
• Ensure middleware, context processors, and authentication backends are compatible with async views.
• Conduct thorough testing to ensure that the asynchronous implementation maintains the current functionality and improves performance.
• Gradual rollout: Start by converting less critical views and monitor for performance gains and potential issues before moving to more critical parts of the application.

Alternatives Considered

The primary alternative to implementing Django's async features is to continue using synchronous views and database operations. This approach relies on the existing, well-understood synchronous programming model, ensuring stability and maintaining the current performance benchmarks. While it lacks the potential performance improvements of async processing, it avoids the complexities and potential challenges involved in refactoring to asynchronous code. This approach would be more about optimizing within the current synchronous framework, focusing on database query optimizations, efficient view logic, and potentially leveraging caching strategies to improve performance.

Additional Context

This transition to asynchronous processing is part of a strategic move towards adopting modern web development practices. It requires careful planning and testing to ensure seamless integration with the existing synchronous parts of the application. The latest version of Django's improved async support makes it a suitable time for this upgrade.

Documentation

For adding Django async support, the official Django documentation provides a comprehensive guide on implementing asynchronous support. Key aspects include:

1. Asynchronous ORM Queries: Django supports async ORM queries, allowing you to run database queries asynchronously. This includes methods with an 'a'-prefix for asynchronous variants and support for async for on all QuerySets.
2. Async Model Methods: Django also supports asynchronous model methods, such as asave and acreate, which can be used for database operations.
3. Performance Considerations: It's important to note the performance implications of running async views under WSGI or sync views under ASGI. Django tries to minimize context-switches between sync and async, but mixing them can lead to a performance penalty.
4. Async Safety: Certain parts of Django, like the ORM, are not safe to use in an async environment due to global state that is not coroutine-aware. To handle this, Django protects async-unsafe parts from execution in an async environment, and you can use sync_to_async() for calling sync functions from an async context.
5. Async Adapter Functions: Django uses adapter functions like async_to_sync() and sync_to_async() from the asgiref.sync module to adapt calling styles between sync and async code.
6. Handling Disconnects: For long-lived requests, an asyncio.CancelledError might be raised if a client disconnects before the view returns a response.

The Django documentation covers these topics and more in-depth and is an essential resource for implementing async in Django. You can read more about asynchronous support in Django on the official documentation.

Additionally, the DEV Community provides an article discussing the comparison between WSGI and ASGI in the context of Django, highlighting the benefits of ASGI for async support, including scalability, concurrency, and real-time functionality. It also covers why ASGI is needed for async support in Django and provides a comparison of Uvicorn and Daphne, two popular ASGI server implementations. You can read more about this on the DEV Community article.

[code-master-ajay]

Hi @brylie, i would like to contribute. Can you tell me how you would like to approach this, because this is a big task and doing it gradually makes more sense.

[brylie]

Yes, this seems like an epic. I suppose a place to start would be to check for feasibility. E.g. check whether we have any synchronous middleware that would force Django to use a thread per request. If this task isn’t feasible, the HTMX task would likely have the most user benefit and could be broken into incremental sub-tasks. #88