Merge remote-tracking branch 'upstream/master'

docs/intro/installation.rst

@@ -5,7 +5,7 @@ Installation
 Ubuntu, Debian or Fedora server
 -------------------------------
 
-**Mayan EDMS** should be deployed_ like any other Django_ project and preferably using virtualenv_.
+**Mayan EDMS** should be deployed like any other Django_ project and preferably using virtualenv_.
 
 If using a Debian_ or Ubuntu_ based Linux distribution getting the executable requirements is as easy as::
 
@@ -15,180 +15,55 @@ If using a Fedora_ based Linux distribution get the executable requirements usin
 
     $ sudo yum install -y git gcc tesseract unpaper python-virtualenv ghostscript libjpeg-turbo-devel libpng-devel poppler-util python-devel
 
-To initialize a ``virtualenv`` to deploy the project do::
+Initialize a ``virtualenv`` to deploy the project::
 
-    $ virtualenv --no-site-packages mayan
+    $ virtualenv --no-site-packages venv
+    $ source venv/bin/activate
 
 Download_ and decompress the latest version of **Mayan EDMS**::
 
-    $ cd mayan
+    $ cd venv
     $ tar -xvzf mayan.tar.gz
 
 Or clone the latest development version straight from github::
 
-    $ cd mayan
+    $ cd venv
-    $ git clone git://github.com/mayan-edms/mayan-edms.git
+    $ git clone https://github.com/mayan-edms/mayan-edms.git
 
 To install the python dependencies ``easy_install`` can be used, however for easier retrieval a production dependencies file is included, to use it execute::
 
-    $ cd mayan
+    $ cd mayan-edms
-    $ source ../bin/activate
     $ pip install -r requirements.txt
 
-Create the database that will hold the data. Install any corresponding python database drivers. Update the settings.py file with you database settings.
+By default **Mayan EDMS** will create a single file SQLite_ database which makes is very easy to start using **Mayan EDMS**.
-If using the ``MySQL`` database manager, use the following commands::
-
-    $ sudo apt-get install libmysqlclient-dev -y
-    $ pip install MySQL-python
-
-If using ``PostgreSQL``, enter the following::
-
-    $ sudo apt-get install libpq-dev -y
-    $ pip install psycopg2
-
-For Fedora systems just use Yum instead of APT::
-
-    $ sudo yum install -y mysql-devel
-    $ pip install MySQL-python
-
-If using ``PostgreSQL``, enter the following::
-
-    $ sudo yum install -y postgresql-devel
-    $ pip install psycopg2
-
 Populate the database with the project's schema doing::
 
     $ ./manage.py syncdb --migrate --noinput
 
-To test your installation, execute Django’s development server using the ``runserver`` command to launch a local instance of Mayan EDMS::
+
+To test your installation, execute Django’s development server using the ``runserver`` command to launch a local instance of **Mayan EDMS**::
 
     $ ./manage.py runserver
 
 Point your browser to http://127:0.0.1:8000, if everything was installed
-correctly you should see the login screen.  After making sure everything
+correctly you should see the login screen and panel showing a randomly generated admin password.
-is running correctly, stop the runserver command and delete the settings_local.py.
-Deploy **Mayan EDMS** using the webserver of your preference. If your are
-using Apache_, a sample site file is included under the contrib directory.
-
-Before finally deploying to your favorite webserver don't forget to collect the
-static files of the project into the ``static`` folder for serving via a webserver::
-
-    $ ./manage.py collectstatic
 
 
-Cloud install
+Production use
--------------
+--------------
-SaaS provied Appsembler_ has started providing a "1-click install" cloud
-offering of **Mayan EDMS**.  Go to their website and click on apps to start
-your trial period of **Mayan EDMS** on the cloud.
 
-Webfaction
+After making sure everything is running correctly, stop the runserver command.
-----------
+Deploy **Mayan EDMS** using the webserver of your preference. For more information
-
+on deployment instructions and examples checkout Django's official documentation
-To install **Mayan EDMS** on Webfaction_, follow these steps:
+on the topic https://docs.djangoproject.com/en/1.6/howto/deployment/
-
-1. Create a new database:
-
-  * Enter the following selections:
-
-    * Type:* ``Mysql``
-    * Name:* ``<username>_mayan``
-    * Encoding:* ``utf-8``
-
-  * Anotate the provided password.
-
-2. Create a new app:
-
-  * Enter the following in the textbox:
-
-    * Name:* ``mayan_app``
-    * App category:* ``mod_wsgi``
-    * App type:* ``mod_wsgi 3.3/Python 2.7``
-
-3. Login via ssh, and execute::
-
-    $ easy_install-2.7 virtualenv
-    $ cd ~/webapps/mayan_app
-    $ virtualenv --no-site-packages mayan
-    $ cd mayan
-    $ git clone git://github.com/mayan-edms/mayan-edms.git
-    $ cd mayan
-    $ source ../bin/activate
-    $ pip install -r requirements.txt
-
-4. Install the Python MySQL database driver::
-
-    $ pip install MySQL-python
-
-5. Create a settings_local.py file, and paste into it the following::
-
-    DATABASES = {
-        'default': {
-            'ENGINE': 'django.db.backends.mysql',
-            'NAME': '<username>_mayan',
-            'USER': '<username>_mayan',
-            'PASSWORD': '<database password from step 1>',
-            'HOST': '',
-            'PORT': '',
-        }
-    }
-
-6. Create the database schema::
-
-    $ ./manage.py syncdb --migrate --noinput
-
-7. Collect the static files of the apps::
-
-    $ ./manage.py collectstatic -l --noinput
-
-8. Create a new app:
-
-  * Enter the following:
-
-    * Name:* ``mayan_static``
-    * App category:* ``Symbolic link``
-    * App type:* ``Symbolic link to static-only app``
-    * Extra info: ``/home/<username>/webapps/mayan_app/mayan/mayan/static``
-
-9. Create the website:
-
-  * Name: ``mayan_edms``
-  * Choose a subdomain
-  * Under ``Site apps:`` enter the following selections:
-
-    * App #1
-
-      * App:* ``mayan_app``
-      * URL path (ex: '/' or '/blog'):* ``/``
-
-    * App #2
-
-      * App:* ``mayan_static``
-      * URL path (ex: '/' or '/blog'):* ``/mayan-static``
-
-10. Edit the file ``~/webapps/mayan_app/apache2/conf/httpd.conf``:
-
-  * Disable the ``DirectoryIndex`` line and the ``DocumentRoot`` line.
-  * Add the following line::
-
-      WSGIScriptAlias / /home/<username>/webapps/mayan_app/mayan/mayan/wsgi/dispatch.wsgi
-
-  * Tune your WSGI process to only use 2 workers (as explained here: `Reducing mod_wsgi Memory Consumption`_)
-    to keep the memory usage under the basic 256MB of RAM provided or upgrade your plan to 512MB,
-    the line that controls the amount of workers launched is::
-
-      WSGIDaemonProcess mayan_app processes=5 python-path=/home/<username>/webapps/mayan_app/lib/python2.7 threads=1
-
-    change it to::
-
-      WSGIDaemonProcess mayan_app processes=2 python-path=/home/<username>/webapps/mayan_app/lib/python2.7 threads=1
 
 
-11. Restart your apache instance:
+Other database managers
+-----------------------
 
-  * Execute::
+If you want to use a database manager other than SQLite_ install any
-
+corresponding python database drivers and create a settings_local.py file
-     apache2/bin/restart
+with the corresponding database settings as shown here: https://docs.djangoproject.com/en/1.6/ref/settings/#std:setting-DATABASES
 
 
 .. _`vendor lock-in`: https://secure.wikimedia.org/wikipedia/en/wiki/Vendor_lock-in
@@ -196,18 +71,11 @@ To install **Mayan EDMS** on Webfaction_, follow these steps:
 .. _Django: http://www.djangoproject.com/
 .. _OCR: https://secure.wikimedia.org/wikipedia/en/wiki/Optical_character_recognition
 .. _`Open source`: https://secure.wikimedia.org/wikipedia/en/wiki/Open_source
-.. _DjangoZoom: http://djangozoom.com/
-.. _Youtube: http://bit.ly/mayan-djangozoom
 .. _Django: http://www.djangoproject.com/
-
-
 .. _Apache: https://www.apache.org/
 .. _Debian: http://www.debian.org/
 .. _Ubuntu: http://www.ubuntu.com/
 .. _Download: https://github.com/mayan-edms/mayan-edms/archives/master
-.. _Webfaction: http://www.webfaction.com
-.. _deployed: https://docs.djangoproject.com/en/1.3/howto/deployment/
 .. _virtualenv: http://www.virtualenv.org/en/latest/index.html
-.. _`Reducing mod_wsgi Memory Consumption`: http://docs.webfaction.com/software/mod-wsgi.html#mod-wsgi-reducing-memory-consumption
 .. _Fedora: http://fedoraproject.org/
-.. _Appsembler: http://appsembler.com/
+.. _SQLite: https://www.sqlite.org/

mayan/apps/app_registry/models.py

@@ -9,7 +9,7 @@ from bootstrap.classes import BootstrapModel, Cleanup
 from navigation.api import register_top_menu
 from project_setup.api import register_setup
 from project_tools.api import register_tool
-from rest_api.classes import EndPoint
+from rest_api.classes import APIEndPoint
 
 logger = logging.getLogger(__name__)
 
@@ -60,10 +60,10 @@ class App(object):
                     logger.debug('version_0_api_services: %s' % version_0_api_services)
 
                     if version_0_api_services:
-                        endpoint = EndPoint(app_name)
+                        api_endpoint = APIEndPoint(app_name)
 
                         for service in version_0_api_services:
-                            endpoint.add_service(**service)
+                            api_endpoint.add_service(**service)
 
     def __unicode__(self):
         return unicode(self.label)

mayan/apps/common/utils.py

@@ -454,11 +454,14 @@ def load_backend(backend_string):
         raise
 
 
-def fs_cleanup(filename):
+def fs_cleanup(filename, suppress_exceptions=True):
     """
     Tries to remove the given filename. Ignores non-existent files
     """
     try:
         os.remove(filename)
     except OSError:
-        pass
+        if suppress_exceptions:
+            pass
+        else:
+            raise

mayan/apps/common/views.py

@@ -1,5 +1,7 @@
 from __future__ import absolute_import
 
+from json import dumps, loads
+
 from django.contrib import messages
 from django.contrib.auth.models import User
 from django.contrib.auth.views import login, password_change
@@ -10,7 +12,6 @@ from django.http import HttpResponseRedirect
 from django.shortcuts import redirect, render_to_response
 from django.template import RequestContext
 from django.utils.http import urlencode
-from django.utils.simplejson import dumps, loads
 from django.utils.translation import ugettext_lazy as _
 from django.views.generic.edit import CreateView, DeleteView, UpdateView
 from django.views.generic.list import ListView

mayan/apps/converter/api.py

@@ -118,8 +118,7 @@ def get_page_count(input_filepath):
 def get_available_transformations_choices():
     result = []
     for transformation in backend.get_available_transformations():
-        transformation_template = u'%s %s' % (TRANSFORMATION_CHOICES[transformation]['label'], u','.join(['<%s>' % argument['name'] if argument['required'] else '[%s]' % argument['name'] for argument in TRANSFORMATION_CHOICES[transformation]['arguments']]))
+        result.append((transformation, TRANSFORMATION_CHOICES[transformation]['label']))
-        result.append([transformation, transformation_template])
 
     return result
 

mayan/apps/documents/__init__.py

@@ -14,6 +14,7 @@ from navigation.api import (register_links, register_top_menu,
     register_model_list_columns, register_multi_item_links,
     register_sidebar_template)
 from project_setup.api import register_setup
+from rest_api.classes import APIEndPoint
 from statistics.classes import StatisticNamespace
 
 from .conf import settings as document_settings
@@ -46,6 +47,7 @@ from .permissions import (
     PERMISSION_DOCUMENT_TRANSFORM, PERMISSION_DOCUMENT_EDIT,
     PERMISSION_DOCUMENT_VERSION_REVERT, PERMISSION_DOCUMENT_NEW_VERSION)
 from .statistics import DocumentStatistics, DocumentUsageStatistics
+from .urls import api_urls
 from .widgets import document_thumbnail
 
 # History setup
@@ -151,3 +153,7 @@ document_search.add_related_field('comments', 'Comment', 'comment', 'object_pk',
 namespace = StatisticNamespace(name='documents', label=_(u'Documents'))
 namespace.add_statistic(DocumentStatistics(name='document_stats', label=_(u'Document tendencies')))
 namespace.add_statistic(DocumentUsageStatistics(name='document_usage', label=_(u'Document usage')))
+
+endpoint = APIEndPoint('documents')
+endpoint.register_urls(api_urls)
+endpoint.add_endpoint('document-list')

mayan/apps/documents/api.py

@@ -17,33 +17,57 @@ from permissions.models import Permission
 from .conf.settings import DISPLAY_SIZE, ZOOM_MAX_LEVEL, ZOOM_MIN_LEVEL
 from .models import Document, DocumentVersion, DocumentPage
 from .permissions import PERMISSION_DOCUMENT_VIEW
-from .resources import ResourceDocument, ResourceDocumentVersion, ResourceDocumentPage
+from .serializers import DocumentSerializer, DocumentVersionSerializer, DocumentPageSerializer
 
 logger = logging.getLogger(__name__)
 
 # API Views
 
 
+class APIDocumentListView(generics.ListAPIView):
+    """
+    Returns a list of all the documents.
+    """
+
+    serializer_class = DocumentSerializer
+    queryset = Document.objects.all()
+
+
 class APIDocumentPageView(generics.RetrieveAPIView):
+    """
+    Returns the selected document page details.
+    """
+
     allowed_methods = ['GET']
-    serializer_class = ResourceDocumentPage
+    serializer_class = DocumentPageSerializer
     queryset = DocumentPage.objects.all()
 
 
 class APIDocumentView(generics.RetrieveAPIView):
+    """
+    Returns the selected document details.
+    """
+
     allowed_methods = ['GET']
-    serializer_class = ResourceDocument
+    serializer_class = DocumentSerializer
     queryset = Document.objects.all()
 
 
 class APIDocumentVersionView(generics.RetrieveAPIView):
+    """
+    Returns the selected document version details.
+    """
+
     allowed_methods = ['GET']
-    serializer_class = DocumentVersion
+    serializer_class = DocumentVersionSerializer
     queryset = DocumentVersion.objects.all()
 
 
-
 class APIDocumentImageView(generics.GenericAPIView):
+    """
+    Returns an image representation of the selected document.
+    """
+
     def get(self, request, pk):
         document = get_object_or_404(Document, pk=pk)
 

mayan/apps/documents/models.py

@@ -143,8 +143,11 @@ class Document(models.Model):
         logger.debug('file_path: %s' % file_path)
 
         if as_base64:
+            mimetype = get_mimetype(open(file_path, 'r'), file_path, mimetype_only=True)[0]
             image = open(file_path, 'r')
-            return u'data:%s;base64,%s' % (get_mimetype(open(file_path, 'r'), file_path, mimetype_only=True)[0], base64.b64encode(image.read()))
+            base64_data = base64.b64encode(image.read())
+            image.close()
+            return u'data:%s;base64,%s' % (mimetype, base64_data)
         else:
             return file_path
 
@@ -562,9 +565,6 @@ class DocumentPage(models.Model):
         verbose_name = _(u'document page')
         verbose_name_plural = _(u'document pages')
 
-    def get_transformation_list(self):
-        return DocumentPageTransformation.objects.get_for_document_page_as_list(self)
-
     @models.permalink
     def get_absolute_url(self):
         return ('document_page_view', [self.pk])
@@ -608,7 +608,7 @@ class DocumentPageTransformation(models.Model):
     document_page = models.ForeignKey(DocumentPage, verbose_name=_(u'document page'))
     order = models.PositiveIntegerField(default=0, blank=True, null=True, verbose_name=_(u'order'), db_index=True)
     transformation = models.CharField(choices=get_available_transformations_choices(), max_length=128, verbose_name=_(u'transformation'))
-    arguments = models.TextField(blank=True, null=True, verbose_name=_(u'arguments'), help_text=_(u'Use dictionaries to indentify arguments, example: %s') % u'{\'degrees\':90}', validators=[ArgumentsValidator()])
+    arguments = models.TextField(blank=True, null=True, verbose_name=_(u'arguments'), help_text=_(u'Use dictionaries to indentify arguments, example: {\'degrees\':90}'), validators=[ArgumentsValidator()])
     objects = DocumentPageTransformationManager()
 
     def __unicode__(self):
@@ -638,3 +638,8 @@ class RecentDocument(models.Model):
         ordering = ('-datetime_accessed',)
         verbose_name = _(u'recent document')
         verbose_name_plural = _(u'recent documents')
+
+
+# Quick hack to break the DocumentPage and DocumentPageTransformation circular dependency
+# Can be remove once the transformations are moved to the converter app
+DocumentPage.add_to_class('get_transformation_list', lambda document_page: DocumentPageTransformation.objects.get_for_document_page_as_list(document_page))

mayan/apps/documents/registry.py

@@ -3,7 +3,6 @@ from __future__ import absolute_import
 from django.conf.urls import url
 
 from .cleanup import cleanup
-from .api import APIDocumentView, APIDocumentVersionView, APIDocumentImageView, APIDocumentPageView
 
 bootstrap_models = [
     {
@@ -16,9 +15,3 @@ bootstrap_models = [
 ]
 cleanup_functions = [cleanup]
 
-version_0_api_services = [
-    {'urlpattern': url(r'^document/(?P<pk>[0-9]+)/$', APIDocumentView.as_view(), name='document-detail'), 'description': 'Show document data', 'url': 'document/<document ID>'},
-    {'urlpattern': url(r'^document_version/(?P<pk>[0-9]+)/$', APIDocumentVersionView.as_view(), name='documentversion-detail'), 'description': '', 'url': 'document_version/<document_version ID>'},
-    {'urlpattern': url(r'^document_page/(?P<pk>[0-9]+)/$', APIDocumentPageView.as_view(), name='documentpage-detail'), 'description': '', 'url': 'document_page/<document page_ID>'},
-    {'urlpattern': url(r'^document/(?P<pk>[0-9]+)/image/$', APIDocumentImageView.as_view(), name='document-image'), 'description': 'Return a base64 image of the document', 'url': 'document/<document_id>/image/?page=<page number>&zoom=<zoom percent>&rotate=<rotation degrees>'},
-]

mayan/apps/documents/resources.py

@@ -1,28 +0,0 @@
-from __future__ import absolute_import
-
-from django.core.urlresolvers import reverse
-
-from rest_framework import serializers
-
-from .models import Document, DocumentVersion, DocumentPage
-
-
-class ResourceDocumentPage(serializers.HyperlinkedModelSerializer):
-    class Meta:
-        model = DocumentPage
-        fields = ('url', 'content', 'page_label', 'page_number')
-
-
-class ResourceDocumentVersion(serializers.HyperlinkedModelSerializer):
-    pages = ResourceDocumentPage(many=True, read_only=True)
-
-    class Meta:
-        model = DocumentVersion
-        fields = ('document', 'major', 'minor', 'micro', 'release_level', 'serial', 'timestamp', 'comment', 'file', 'mimetype', 'encoding', 'filename', 'checksum', 'pages')
-
-
-class ResourceDocument(serializers.HyperlinkedModelSerializer):
-    versions = ResourceDocumentVersion(many=True, read_only=True)
-
-    class Meta:
-        model = Document

mayan/apps/documents/serializers.py

@@ -0,0 +1,26 @@
+from __future__ import absolute_import
+
+from django.core.urlresolvers import reverse
+
+from rest_framework import serializers
+
+from .models import Document, DocumentVersion, DocumentPage
+
+
+class DocumentPageSerializer(serializers.HyperlinkedModelSerializer):
+    class Meta:
+        model = DocumentPage
+
+
+class DocumentVersionSerializer(serializers.HyperlinkedModelSerializer):
+    pages = DocumentPageSerializer(many=True, read_only=True)
+
+    class Meta:
+        model = DocumentVersion
+
+
+class DocumentSerializer(serializers.HyperlinkedModelSerializer):
+    versions = DocumentVersionSerializer(many=True, read_only=True)
+
+    class Meta:
+        model = Document

mayan/apps/documents/templates/document_print.html

@@ -1,5 +1,5 @@
 {% load project_tags %}
-{#{% load printing_tags %}#}
+
 <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN"
 "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
 <html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
@@ -13,7 +13,7 @@
                 padding: 0;
                 margin: 0;
                 background: #fff;
-                color: #000;               
+                color: #000;
             }
             html, body {
                 height: 100%;
@@ -25,7 +25,7 @@
             }
             * html .container {
                 height: 100%;
-            }          
+            }
             .centered {
                   position: absolute;
                   top: 0;
@@ -33,7 +33,7 @@
                   bottom: 0;
                   left: 0;
                   margin: auto;
-            } 
+            }
             {% endcomment %}
             .break { page-break-after: always; }
             img { border: 1px solid black; }
@@ -45,15 +45,12 @@
                 margin-right: auto;
             }
         </style>
-	</head>
+    </head>
-	<body>
+    <body>
         {% for page in pages %}
-            {#{% get_document_size object %}#}
             <div class="{% if forloop.counter > 1 and not forloop.last %}break{% endif %}">
-                {#<img src="{% url 'document_display_print' object.id %}?page={{ page.page_number }}" {% if document_aspect > page_aspect %}width="97%"{% else %}height="97%"{% endif %} />#}
                 <img src="{% url 'document_display_print' object.id %}?page={{ page.page_number }}" />
             </div>
         {% endfor %}
-
     </body>
 </html>

mayan/apps/documents/urls.py

@@ -2,8 +2,9 @@ from __future__ import absolute_import
 
 from django.conf.urls import patterns, url
 
-from .conf.settings import (PREVIEW_SIZE, PRINT_SIZE, THUMBNAIL_SIZE,
+from .api import APIDocumentView, APIDocumentListView, APIDocumentVersionView, APIDocumentImageView, APIDocumentPageView
-    DISPLAY_SIZE, MULTIPAGE_PREVIEW_SIZE)
+from .conf.settings import (PREVIEW_SIZE, PRINT_SIZE, DISPLAY_SIZE,
+    MULTIPAGE_PREVIEW_SIZE)
 
 urlpatterns = patterns('documents.views',
     url(r'^list/$', 'document_list', (), 'document_list'),
@@ -19,14 +20,9 @@ urlpatterns = patterns('documents.views',
 
     url(r'^(?P<document_id>\d+)/display/preview/$', 'get_document_image', {'size': PREVIEW_SIZE}, 'document_preview'),
     url(r'^(?P<document_id>\d+)/display/preview/multipage/$', 'get_document_image', {'size': MULTIPAGE_PREVIEW_SIZE}, 'document_preview_multipage'),
-    url(r'^(?P<document_id>\d+)/display/thumbnail/$', 'get_document_image', {'size': THUMBNAIL_SIZE}, 'document_thumbnail'),
     url(r'^(?P<document_id>\d+)/display/$', 'get_document_image', {'size': DISPLAY_SIZE}, 'document_display'),
     url(r'^(?P<document_id>\d+)/display/print/$', 'get_document_image', {'size': PRINT_SIZE}, 'document_display_print'),
 
-    url(r'^(?P<document_id>\d+)/display/preview/base64/$', 'get_document_image', {'size': PREVIEW_SIZE, 'base64_version': True}, 'document_preview_base64'),
-    url(r'^(?P<document_id>\d+)/display/preview/multipage/base64/$', 'get_document_image', {'size': MULTIPAGE_PREVIEW_SIZE, 'base64_version': True}, 'document_preview_multipage_base64'),
-    url(r'^(?P<document_id>\d+)/display/thumbnail/base64/$', 'get_document_image', {'size': THUMBNAIL_SIZE, 'base64_version': True}, 'document_thumbnail_base64'),
-
     url(r'^(?P<document_id>\d+)/download/$', 'document_download', (), 'document_download'),
     url(r'^multiple/download/$', 'document_multiple_download', (), 'document_multiple_download'),
     url(r'^(?P<document_id>\d+)/find_duplicates/$', 'document_find_duplicates', (), 'document_find_duplicates'),
@@ -72,5 +68,12 @@ urlpatterns = patterns('documents.views',
     url(r'^type/filename/(?P<document_type_filename_id>\d+)/edit/$', 'document_type_filename_edit', (), 'document_type_filename_edit'),
     url(r'^type/filename/(?P<document_type_filename_id>\d+)/delete/$', 'document_type_filename_delete', (), 'document_type_filename_delete'),
     url(r'^type/(?P<document_type_id>\d+)/filename/create/$', 'document_type_filename_create', (), 'document_type_filename_create'),
+)
 
+api_urls = patterns('',
+    url(r'^documents/$', APIDocumentListView.as_view(), name='document-list'),
+    url(r'^documents/(?P<pk>[0-9]+)/$', APIDocumentView.as_view(), name='document-detail'),
+    url(r'^document_version/(?P<pk>[0-9]+)/$', APIDocumentVersionView.as_view(), name='documentversion-detail'),
+    url(r'^document_page/(?P<pk>[0-9]+)/$', APIDocumentPageView.as_view(), name='documentpage-detail'),
+    url(r'^documents/(?P<pk>[0-9]+)/image/$', APIDocumentImageView.as_view(), name='document-image'),
 )

mayan/apps/main/templates/base.html

@@ -143,24 +143,37 @@
             });
 
             $("a.fancybox").fancybox({
-                openEffect  : 'elastic',
+                'openEffect'  : 'elastic',
-                closeEffect : 'elastic',
+                'closeEffect' : 'elastic',
                 prevEffect  : 'none',
                 nextEffect  : 'none',
                 'titleShow' : true,
                 'type'      : 'image',
-                'autoResize': true
+                'autoResize': true,
             });
 
-            $("a.fancybox-staging").fancybox({
+            $("a.fancybox-staging").click(function(e) {
-                openEffect  : 'elastic',
+                var $this = $(this);
-                closeEffect : 'elastic',
+
-                prevEffect  : 'none',
+                    $.get($this.attr('href'), function( result ) {
-                nextEffect  : 'none',
+                        if (result.status == 'success') {
-                'titleShow' : true,
+                            $.fancybox.open([
-                'type'      : 'image',
+                                {
-                'autoResize': true
+                                    href : result.data,
-            });
+                                    title : $this.attr('title'),
+                                    'openEffect'  : 'elastic',
+                                    'closeEffect' : 'elastic',
+                                    prevEffect  : 'none',
+                                    nextEffect  : 'none',
+                                    'titleShow' : true,
+                                    'type'      : 'image',
+                                    'autoResize': true,
+                                },
+                            ]);
+                        }
+                    })
+                e.preventDefault();
+            })
 
             $("a.fancybox-noscaling").fancybox({
                 openEffect  : 'elastic',

mayan/apps/rest_api/classes.py

@@ -1,7 +1,9 @@
 from __future__ import absolute_import
 
+from django.conf.urls import include, patterns, url
 
-class EndPoint(object):
+
+class APIEndPoint(object):
     _registry = {}
 
     @classmethod
@@ -17,14 +19,21 @@ class EndPoint(object):
 
     def __init__(self, name):
         self.name = name
-        self.services = []
+        self.endpoints = []
         self.__class__._registry[name] = self
 
-    def add_service(self, urlpattern, url=None, description=None):
+    def add_endpoint(self, view_name, description=None):
-        self.services.append(
+        self.endpoints.append(
             {
                 'description': description,
-                'url': url,
+                'view_name': view_name,
-                'urlpattern': urlpattern,
             }
         )
+
+    def register_urls(self, urlpatterns):
+        from .urls import version_0_urlpatterns
+        endpoint_urls = patterns('',
+            url(r'^%s/' % self.name, include(urlpatterns)),
+        )
+
+        version_0_urlpatterns += endpoint_urls

mayan/apps/rest_api/registry.py

@@ -2,4 +2,4 @@ from __future__ import absolute_import
 
 from .links import link_api
 
-setup_links = [link_api]
+tool_links = [link_api]

mayan/apps/rest_api/urls.py

@@ -2,23 +2,14 @@ from __future__ import absolute_import
 
 from django.conf.urls import include, patterns, url
 
-from .classes import EndPoint
+from .views import APIBase, Version_0, APIAppView
-from .views import APIBase, Version_0, EndPointView
 
-version_0_endpoints_urlpatterns = patterns('',
+version_0_urlpatterns = patterns('',
     url(r'^$', Version_0.as_view(), name='api-version-0'),
-    url(r'^(?P<endpoint_name>\w+)$', EndPointView.as_view(), name='api-version-0-endpoint'),
+    url(r'^(?P<app_name>\w+)$', APIAppView.as_view(), name='api-version-0-app'),
 )
 
-for endpoint in EndPoint.get_all():
-    endpoint_urlpatterns = patterns('')
-
-    for service in endpoint.services:
-        endpoint_urlpatterns += patterns('', service['urlpattern'])
-
-    version_0_endpoints_urlpatterns += patterns('', url(r'^%s/' % endpoint.name, include(endpoint_urlpatterns)))
-
 urlpatterns = patterns('',
     url(r'^$', APIBase.as_view(), name='api-root'),
-    url(r'^v0/', include(version_0_endpoints_urlpatterns)),
+    url(r'^v0/', include(version_0_urlpatterns)),
 )

mayan/apps/rest_api/views.py

@@ -10,7 +10,7 @@ from rest_framework.generics import RetrieveAPIView
 from rest_framework.response import Response
 from rest_framework.reverse import reverse
 
-from .classes import EndPoint
+from .classes import APIEndPoint
 
 logger = logging.getLogger(__name__)
 
@@ -19,6 +19,10 @@ registered_version_0_endpoints = [
 
 
 class APIBase(generics.GenericAPIView):
+    """
+    Main entry point of the API.
+    """
+
     def get(self, request, format=None):
         return Response([
             {'name': 'Version 0', 'url': reverse('api-version-0', request=request, format=format)}
@@ -26,27 +30,35 @@ class APIBase(generics.GenericAPIView):
 
 
 class Version_0(generics.GenericAPIView):
+    """
+    API version 0 entry points.
+    """
+
     def get(self, request, format=None):
         return Response({
-            'endpoints': [
+            'apps': [
-                {'name': unicode(endpoint), 'url': reverse('api-version-0-endpoint', args=[unicode(endpoint)], request=request, format=format)} for endpoint in EndPoint.get_all()
+                {'name': unicode(endpoint), 'url': reverse('api-version-0-app', args=[unicode(endpoint)], request=request, format=format)} for endpoint in APIEndPoint.get_all()
             ],
         })
 
-class EndPointView(generics.GenericAPIView):
+
-    def get(self, request, endpoint_name, format=None):
+class APIAppView(generics.GenericAPIView):
+    """
+    Entry points of the selected app.
+    """
+
+    def get(self, request, app_name, format=None):
         result = []
 
-        endpoint = EndPoint.get(endpoint_name)
+        api_app = APIEndPoint.get(app_name)
-        for service in endpoint.services:
+        for endpoint in api_app.endpoints:
             result.append(
                 {
-                    'description': service['description'],
+                    'description': endpoint['description'],
-                    'name': service['urlpattern'].name,
+                    'url': reverse(endpoint['view_name'], request=request, format=format),
-                    'url': service['url'],
                 }
             )
 
         return Response({
-            'services': result
+            'endpoints': result
         })

mayan/apps/sources/__init__.py

@@ -6,7 +6,9 @@ from common.utils import encapsulate
 from documents.models import Document
 from navigation.api import register_links, register_model_list_columns
 from project_setup.api import register_setup
+from rest_api.classes import APIEndPoint
 
+from .classes import StagingFile
 from .links import (document_create_multiple, document_create_siblings,
     staging_file_delete, setup_sources, setup_web_form_list,
     setup_staging_folder_list, setup_watch_folder_list,
@@ -16,23 +18,20 @@ from .links import (document_create_multiple, document_create_siblings,
     upload_version)
 from .models import (WebForm, StagingFolder, SourceTransformation,
     WatchFolder)
-from .staging import StagingFile
+from .urls import api_urls
 from .widgets import staging_file_thumbnail
 
-register_links(StagingFile, [staging_file_delete])
+register_links([StagingFile], [staging_file_delete])
 
 register_links(SourceTransformation, [setup_source_transformation_edit, setup_source_transformation_delete])
 
-# register_links(['setup_web_form_list', 'setup_staging_folder_list', 'setup_watch_folder_list', 'setup_source_create'], [setup_web_form_list, setup_staging_folder_list, setup_watch_folder_list], menu_name='form_header')
 register_links(['setup_web_form_list', 'setup_staging_folder_list', 'setup_watch_folder_list', 'setup_source_create'], [setup_web_form_list, setup_staging_folder_list], menu_name='form_header')
 
-# register_links(WebForm, [setup_web_form_list, setup_staging_folder_list, setup_watch_folder_list], menu_name='form_header')
 register_links(WebForm, [setup_web_form_list, setup_staging_folder_list], menu_name='form_header')
 register_links(WebForm, [setup_source_transformation_list, setup_source_edit, setup_source_delete])
 
 register_links(['setup_web_form_list', 'setup_staging_folder_list', 'setup_watch_folder_list', 'setup_source_edit', 'setup_source_delete', 'setup_source_create'], [setup_sources, setup_source_create], menu_name='sidebar')
 
-# register_links(StagingFolder, [setup_web_form_list, setup_staging_folder_list, setup_watch_folder_list], menu_name='form_header')
 register_links(StagingFolder, [setup_web_form_list, setup_staging_folder_list], menu_name='form_header')
 register_links(StagingFolder, [setup_source_transformation_list, setup_source_edit, setup_source_delete])
 
@@ -48,7 +47,7 @@ source_views = ['setup_web_form_list', 'setup_staging_folder_list', 'setup_watch
 
 register_model_list_columns(StagingFile, [
         {'name': _(u'thumbnail'), 'attribute':
-            encapsulate(lambda x: staging_file_thumbnail(x))
+            encapsulate(lambda x: staging_file_thumbnail(x, gallery_name='staging_list', title=x.filename, size='100'))
         },
     ])
 
@@ -56,3 +55,7 @@ register_setup(setup_sources)
 
 register_links([Document, 'document_list_recent', 'document_list', 'document_create', 'document_create_multiple', 'upload_interactive', 'staging_file_delete'], [document_create_multiple], menu_name='secondary_menu')
 register_links(Document, [document_create_siblings])
+
+endpoint = APIEndPoint('sources')
+endpoint.register_urls(api_urls)
+endpoint.add_endpoint('stagingfolder-list')

mayan/apps/sources/api.py

@@ -0,0 +1,81 @@
+from __future__ import absolute_import
+
+import logging
+
+from django.shortcuts import get_object_or_404
+
+from rest_framework import generics
+from rest_framework.response import Response
+
+from converter.exceptions import UnkownConvertError, UnknownFileFormat
+from converter.literals import DEFAULT_PAGE_NUMBER, DEFAULT_ROTATION, DEFAULT_ZOOM_LEVEL
+from documents.conf.settings import DISPLAY_SIZE, ZOOM_MAX_LEVEL, ZOOM_MIN_LEVEL
+
+from .models import StagingFolder
+from .serializers import SerializerStagingFolder, SerializerStagingFolderFile
+
+logger = logging.getLogger(__name__)
+
+# API Views
+
+
+class APIStagingSourceFileView(generics.GenericAPIView):
+    """
+    Details of the selected staging file.
+    """
+    def get(self, request, staging_folder_pk, filename):
+        staging_folder = get_object_or_404(StagingFolder, pk=staging_folder_pk)
+        return Response(SerializerStagingFolderFile(staging_folder.get_file(encoded_filename=filename), context={'request': request}).data)
+
+
+class APIStagingSourceListView(generics.ListAPIView):
+    """
+    Returns a list of all the staging folders and the files they contain.
+    """
+
+    serializer_class = SerializerStagingFolder
+    queryset = StagingFolder.objects.all()
+
+
+class APIStagingSourceView(generics.RetrieveAPIView):
+    """
+    Details of the selected staging folders and the files it contains.
+    """
+    serializer_class = SerializerStagingFolder
+    queryset = StagingFolder.objects.all()
+
+
+class APIStagingSourceFileImageView(generics.GenericAPIView):
+    """
+    Image of the selected staging file.
+    """
+    def get(self, request, staging_folder_pk, filename):
+        staging_folder = get_object_or_404(StagingFolder, pk=staging_folder_pk)
+        staging_file = staging_folder.get_file(encoded_filename=filename)
+
+        size = request.GET.get('size', DISPLAY_SIZE)
+
+        page = int(request.GET.get('page', DEFAULT_PAGE_NUMBER))
+
+        zoom = int(request.GET.get('zoom', DEFAULT_ZOOM_LEVEL))
+
+        if request.GET.get('as_base64', False):
+            base64_version = True
+
+        if zoom < ZOOM_MIN_LEVEL:
+            zoom = ZOOM_MIN_LEVEL
+
+        if zoom > ZOOM_MAX_LEVEL:
+            zoom = ZOOM_MAX_LEVEL
+
+        rotation = int(request.GET.get('rotation', DEFAULT_ROTATION)) % 360
+
+        try:
+            return Response({'status': 'success',
+                'data': staging_file.get_image(size=size, page=page, zoom=zoom, rotation=rotation, as_base64=True)
+                })
+        except UnknownFileFormat as exception:
+            return Response({'status': 'error', 'detail': 'unknown_file_format', 'message': unicode(exception)})
+        except UnkownConvertError as exception:
+            return Response({'status': 'error', 'detail': 'converter_error', 'message': unicode(exception)})
+

mayan/apps/sources/classes.py

@@ -0,0 +1,50 @@
+from __future__ import absolute_import
+
+import base64
+import os
+import urllib
+
+from django.core.files import File
+
+from converter.api import convert
+from mimetype.api import get_mimetype
+
+
+class StagingFile(object):
+    """
+    Simple class to extend the File class to add preview capabilities
+    files in a directory on a storage
+    """
+    def __init__(self, staging_folder, filename=None, encoded_filename=None):
+        self.staging_folder = staging_folder
+        if encoded_filename:
+            self.encoded_filename = str(encoded_filename)
+            self.filename = base64.urlsafe_b64decode(urllib.unquote_plus(self.encoded_filename))
+        else:
+            self.filename = filename
+            self.encoded_filename = base64.urlsafe_b64encode(filename)
+
+    def __unicode__(self):
+        return unicode(self.filename)
+
+    def as_file(self):
+        return File(file=open(self.get_full_path(), mode='rb'), name=self.filename)
+
+    def get_full_path(self):
+        return os.path.join(self.staging_folder.folder_path, self.filename)
+
+    def get_image(self, size, page, zoom, rotation, as_base64=True):
+        # TODO: add support for transformations
+        converted_file_path = convert(self.get_full_path(), size=size)
+
+        if as_base64:
+            mimetype = get_mimetype(open(converted_file_path, 'r'), converted_file_path, mimetype_only=True)[0]
+            image = open(converted_file_path, 'r')
+            base64_data = base64.b64encode(image.read())
+            image.close()
+            return u'data:%s;base64,%s' % (mimetype, base64_data)
+        else:
+            return converted_file_path
+
+    def delete(self):
+        os.unlink(self.get_full_path())

mayan/apps/sources/forms.py

@@ -1,5 +1,7 @@
 from __future__ import absolute_import
 
+import logging
+
 from django import forms
 from django.utils.translation import ugettext_lazy as _
 from django.utils.translation import ugettext
@@ -11,6 +13,8 @@ from .models import (WebForm, StagingFolder, SourceTransformation,
 from .widgets import FamFamRadioSelect
 from .utils import validate_whitelist_blacklist
 
+logger = logging.getLogger(__name__)
+
 
 class StagingDocumentForm(DocumentForm):
     """
@@ -18,15 +22,16 @@ class StagingDocumentForm(DocumentForm):
     StagingFile class passed as 'cls' argument
     """
     def __init__(self, *args, **kwargs):
-        cls = kwargs.pop('cls')
         show_expand = kwargs.pop('show_expand', False)
         self.source = kwargs.pop('source')
         super(StagingDocumentForm, self).__init__(*args, **kwargs)
+
         try:
             self.fields['staging_file_id'].choices = [
-                (staging_file.id, staging_file) for staging_file in cls.get_all()
+                (staging_file.encoded_filename, unicode(staging_file)) for staging_file in self.source.get_files()
             ]
-        except:
+        except Exception as exception:
+            logger.error('exception: %s' % exception)
             pass
 
         if show_expand:

mayan/apps/sources/links.py

@@ -13,8 +13,7 @@ from .permissions import (PERMISSION_SOURCES_SETUP_VIEW,
 document_create_multiple = {'text': _(u'upload new documents'), 'view': 'document_create_multiple', 'famfam': 'page_add', 'permissions': [PERMISSION_DOCUMENT_CREATE], 'children_view_regex': [r'upload_interactive']}
 document_create_siblings = {'text': _(u'clone metadata'), 'view': 'document_create_siblings', 'args': 'object.id', 'famfam': 'page_copy', 'permissions': [PERMISSION_DOCUMENT_CREATE]}
 
-staging_file_preview = {'text': _(u'preview'), 'class': 'fancybox-noscaling', 'view': 'staging_file_preview', 'args': ['source.source_type', 'source.pk', 'object.id'], 'famfam': 'zoom', 'permissions': [PERMISSION_DOCUMENT_NEW_VERSION, PERMISSION_DOCUMENT_CREATE]}
+staging_file_delete = {'text': _(u'delete'), 'view': 'staging_file_delete', 'args': ['source.pk', 'object.encoded_filename'], 'famfam': 'delete', 'keep_query': True, 'permissions': [PERMISSION_DOCUMENT_NEW_VERSION, PERMISSION_DOCUMENT_CREATE]}
-staging_file_delete = {'text': _(u'delete'), 'view': 'staging_file_delete', 'args': ['source.source_type', 'source.pk', 'object.id'], 'famfam': 'delete', 'keep_query': True, 'permissions': [PERMISSION_DOCUMENT_NEW_VERSION, PERMISSION_DOCUMENT_CREATE]}
 
 setup_sources = {'text': _(u'sources'), 'view': 'setup_web_form_list', 'famfam': 'application_form', 'icon': 'application_form.png', 'children_classes': [WebForm], 'permissions': [PERMISSION_SOURCES_SETUP_VIEW], 'children_view_regex': [r'setup_web_form', r'setup_staging_folder', r'setup_source_']}
 setup_web_form_list = {'text': _(u'web forms'), 'view': 'setup_web_form_list', 'famfam': 'application_form', 'icon': 'application_form.png', 'children_classes': [WebForm], 'permissions': [PERMISSION_SOURCES_SETUP_VIEW]}

mayan/apps/sources/models.py

@@ -2,6 +2,7 @@ from __future__ import absolute_import
 
 from ast import literal_eval
 import logging
+import os
 
 from django.contrib.contenttypes import generic
 from django.contrib.contenttypes.models import ContentType
@@ -20,6 +21,7 @@ from history.api import create_history
 from metadata.api import save_metadata_list
 from scheduler.api import register_interval_job, remove_job
 
+from .classes import StagingFile
 from .literals import (SOURCE_CHOICES, SOURCE_CHOICES_PLURAL,
     SOURCE_INTERACTIVE_UNCOMPRESS_CHOICES, SOURCE_CHOICE_WEB_FORM,
     SOURCE_CHOICE_STAGING, SOURCE_ICON_DISK, SOURCE_ICON_DRIVE,
@@ -166,6 +168,16 @@ class StagingFolder(InteractiveBaseModel):
 
         return DIMENSION_SEPARATOR.join(dimensions)
 
+    def get_file(self, *args, **kwargs):
+        return StagingFile(staging_folder=self, *args, **kwargs)
+
+    def get_files(self):
+        try:
+            for entry in sorted([os.path.normcase(f) for f in os.listdir(self.folder_path) if os.path.isfile(os.path.join(self.folder_path, f))]):
+                yield self.get_file(filename=entry)
+        except OSError as exception:
+            raise Exception(_(u'Unable get list of staging files: %s') % exception)
+
     class Meta(InteractiveBaseModel.Meta):
         verbose_name = _(u'staging folder')
         verbose_name_plural = _(u'staging folders')

mayan/apps/sources/registry.py

@@ -1,6 +1,7 @@
 from __future__ import absolute_import
 
+from django.conf.urls import url
+
 from .cleanup import cleanup
 
-
 cleanup_functions = [cleanup]

mayan/apps/sources/serializers.py

@@ -0,0 +1,36 @@
+from __future__ import absolute_import
+
+import logging
+
+from rest_framework import serializers
+from rest_framework.reverse import reverse
+
+from .models import StagingFolder
+
+logger = logging.getLogger(__name__)
+
+
+class SerializerStagingFolderFile(serializers.Serializer):
+    url = serializers.SerializerMethodField('get_url')
+    image_url = serializers.SerializerMethodField('get_image_url')
+    filename = serializers.CharField(max_length=255)
+
+    def get_url(self, obj):
+        return reverse('stagingfolderfile-detail', args=[obj.staging_folder.pk, obj.encoded_filename], request=self.context.get('request'))
+
+    def get_image_url(self, obj):
+        return reverse('stagingfolderfile-image-view', args=[obj.staging_folder.pk, obj.encoded_filename], request=self.context.get('request'))
+
+
+class SerializerStagingFolder(serializers.HyperlinkedModelSerializer):
+    files = serializers.SerializerMethodField('get_files')
+
+    def get_files(self, obj):
+        try:
+            return [SerializerStagingFolderFile(entry, context=self.context).data for entry in obj.get_files()]
+        except Exception as exception:
+            logger.error('unhandled exception: %s' % exception)
+            return []
+
+    class Meta:
+        model = StagingFolder

mayan/apps/sources/staging.py

@@ -1,134 +0,0 @@
-from __future__ import absolute_import
-
-import errno
-import hashlib
-import os
-
-from django.core.exceptions import ObjectDoesNotExist
-from django.core.files.base import File
-from django.utils.encoding import smart_str
-from django.utils.translation import ugettext
-
-from converter.api import convert, cache_cleanup
-from converter.exceptions import UnknownFileFormat, UnkownConvertError
-from documents.conf.settings import THUMBNAIL_SIZE
-from mimetype.api import (get_icon_file_path, get_error_icon_file_path,
-    get_mimetype)
-
-
-DEFAULT_STAGING_DIRECTORY = u'/tmp'
-
-HASH_FUNCTION = lambda x: hashlib.sha256(x).hexdigest()
-
-
-def get_all_files(path):
-    try:
-        return sorted([os.path.normcase(f) for f in os.listdir(path) if os.path.isfile(os.path.join(path, f))])
-    except OSError, exc:
-        raise Exception(ugettext(u'Unable get list of staging files: %s') % exc)
-
-
-def _return_new_class():
-    return type('StagingFile', (StagingFile,), dict(StagingFile.__dict__))
-
-
-def create_staging_file_class(request, directory_path, source=None):
-    cls = _return_new_class()
-    # cls.set_path(evaluate_user_staging_path(request, source))
-    cls.set_path(directory_path)
-    if source is not None:
-        cls.set_source(source)
-    return cls
-
-
-class StagingFile(object):
-    """
-    Simple class to encapsulate the files in a directory and hide the
-    specifics to the view
-    """
-    path = DEFAULT_STAGING_DIRECTORY
-    source = None
-
-    @classmethod
-    def set_path(cls, path):
-        cls.path = path
-
-    @classmethod
-    def set_source(cls, source):
-        cls.source = source
-
-    @classmethod
-    def get_all(cls):
-        """
-        Return a list of StagingFile instances corresponding to the
-        current path
-        """
-        staging_files = []
-        for filename in get_all_files(cls.path):
-            staging_files.append(StagingFile(
-                filepath=os.path.join(cls.path, filename), source=cls.source))
-
-        return staging_files
-
-    @classmethod
-    def get(cls, id):
-        """
-        Return a single StagingFile instance corresponding to the id
-        given as argument
-        """
-        files_dict = dict([(file.id, file) for file in cls.get_all()])
-        if id in files_dict:
-            return files_dict[id]
-        else:
-            raise ObjectDoesNotExist
-
-    def __init__(self, filepath, source=None):
-        self.source = source
-        self.filepath = filepath
-        self.filename = os.path.basename(filepath)
-        self._id = HASH_FUNCTION(smart_str(filepath))
-
-    def __unicode__(self):
-        return self.filename
-
-    def __repr__(self):
-        return self.__unicode__()
-
-    def __getattr__(self, name):
-        if name == 'id':
-            return self._id
-        else:
-            raise AttributeError
-
-    def upload(self):
-        """
-        Return a StagingFile encapsulated in a File class instance to
-        allow for easier upload of staging files
-        """
-        try:
-            return File(file(self.filepath, 'rb'), name=self.filename)
-        except Exception, exc:
-            raise Exception(ugettext(u'Unable to upload staging file: %s') % exc)
-
-    def delete(self, preview_size, transformations):
-        cache_cleanup(self.filepath, size=preview_size, transformations=transformations)
-        try:
-            os.unlink(self.filepath)
-        except OSError, exc:
-            if exc.errno == errno.ENOENT:
-                pass
-            else:
-                raise Exception(ugettext(u'Unable to delete staging file: %s') % exc)
-
-    def get_valid_image(self, size=THUMBNAIL_SIZE, transformations=None):
-        return convert(self.filepath, size=size, cleanup_files=False, transformations=transformations)
-
-    def get_image(self, size, transformations):
-        try:
-            return self.get_valid_image(size=size, transformations=transformations)
-            # return convert(self.filepath, size=size, cleanup_files=False, transformations=transformations)
-        except UnknownFileFormat:
-            mimetype, encoding = get_mimetype(open(self.filepath, 'rb'), self.filepath)
-            return get_icon_file_path(mimetype)
-        except UnkownConvertError:
-            return get_error_icon_file_path()

mayan/apps/sources/urls.py

@@ -2,14 +2,14 @@ from __future__ import absolute_import
 
 from django.conf.urls import patterns, url
 
+from .api import (APIStagingSourceListView, APIStagingSourceView,
+    APIStagingSourceFileView, APIStagingSourceFileImageView)
 from .literals import (SOURCE_CHOICE_WEB_FORM, SOURCE_CHOICE_STAGING,
     SOURCE_CHOICE_WATCH)
 from .wizards import DocumentCreateWizard
 
 urlpatterns = patterns('sources.views',
-    url(r'^staging_file/type/(?P<source_type>\w+)/(?P<source_id>\d+)/(?P<staging_file_id>\w+)/preview/$', 'staging_file_preview', (), 'staging_file_preview'),
+    url(r'^staging_file/(?P<staging_folder_pk>\d+)/(?P<filename>.+)/delete/$', 'staging_file_delete', name='staging_file_delete'),
-    url(r'^staging_file/type/(?P<source_type>\w+)/(?P<source_id>\d+)/(?P<staging_file_id>\w+)/delete/$', 'staging_file_delete', (), 'staging_file_delete'),
-    url(r'^staging_file/type/staging_folder/(?P<source_id>\d+)/(?P<staging_file_id>\w+)/thumbnail/$', 'staging_file_thumbnail', (), 'staging_file_thumbnail'),
 
     url(r'^upload/document/new/interactive/(?P<source_type>\w+)/(?P<source_id>\d+)/$', 'upload_interactive', (), 'upload_interactive'),
     url(r'^upload/document/new/interactive/$', 'upload_interactive', (), 'upload_interactive'),
@@ -38,3 +38,10 @@ urlpatterns = patterns('sources.views',
     url(r'^create/from/local/multiple/$', DocumentCreateWizard.as_view(), name='document_create_multiple'),
     url(r'^(?P<document_id>\d+)/create/siblings/$', 'document_create_siblings', (), 'document_create_siblings'),
 )
+
+api_urls = patterns('',
+    url(r'^staging_folders/file/(?P<staging_folder_pk>[0-9]+)/(?P<filename>.+)/image/$', APIStagingSourceFileImageView.as_view(), name='stagingfolderfile-image-view'),
+    url(r'^staging_folders/file/(?P<staging_folder_pk>[0-9]+)/(?P<filename>.+)/$', APIStagingSourceFileView.as_view(), name='stagingfolderfile-detail'),
+    url(r'^staging_folders/$', APIStagingSourceListView.as_view(), name='stagingfolder-list'),
+    url(r'^staging_folders/(?P<pk>[0-9]+)/$', APIStagingSourceView.as_view(), name='stagingfolder-detail')
+)

mayan/apps/sources/views.py

@@ -12,11 +12,8 @@ from django.utils.safestring import mark_safe
 from django.utils.translation import ugettext
 from django.utils.translation import ugettext_lazy as _
 
-import sendfile
-
 from acls.models import AccessEntry
 from common.utils import encapsulate
-from documents.conf.settings import THUMBNAIL_SIZE
 from documents.exceptions import NewDocumentVersionNotAllowed
 from documents.models import DocumentType, Document
 from documents.permissions import (PERMISSION_DOCUMENT_CREATE,
@@ -34,7 +31,6 @@ from .models import (WebForm, StagingFolder, SourceTransformation,
 from .permissions import (PERMISSION_SOURCES_SETUP_VIEW,
     PERMISSION_SOURCES_SETUP_EDIT, PERMISSION_SOURCES_SETUP_DELETE,
     PERMISSION_SOURCES_SETUP_CREATE)
-from .staging import create_staging_file_class
 
 
 def document_create_siblings(request, document_id):
@@ -219,17 +215,17 @@ def upload_interactive(request, source_type=None, source_id=None, document_pk=No
         elif source_type == SOURCE_CHOICE_STAGING:
             staging_folder = get_object_or_404(StagingFolder, pk=source_id)
             context['source'] = staging_folder
-            StagingFile = create_staging_file_class(request, staging_folder.folder_path, source=staging_folder)
+
             if request.method == 'POST':
                 form = StagingDocumentForm(request.POST, request.FILES,
-                    cls=StagingFile, document_type=document_type,
+                    document_type=document_type,
                     show_expand=(staging_folder.uncompress == SOURCE_UNCOMPRESS_CHOICE_ASK) and not document,
                     source=staging_folder,
                     instance=document
                 )
                 if form.is_valid():
                     try:
-                        staging_file = StagingFile.get(form.cleaned_data['staging_file_id'])
+                        staging_file = staging_folder.get_file(encoded_filename=form.cleaned_data['staging_file_id'])
                         if document:
                             expand = False
                         else:
@@ -244,7 +240,7 @@ def upload_interactive(request, source_type=None, source_id=None, document_pk=No
                         new_filename = get_form_filename(form)
 
                         result = staging_folder.upload_file(
-                            staging_file.upload(),
+                            staging_file.as_file(),
                             new_filename, use_file_name=form.cleaned_data.get('use_file_name', False),
                             document_type=document_type,
                             expand=expand,
@@ -266,8 +262,7 @@ def upload_interactive(request, source_type=None, source_id=None, document_pk=No
                             messages.warning(request, _(u'Staging file: %s, was not compressed, uploaded as a single file.') % staging_file.filename)
 
                         if staging_folder.delete_after_upload:
-                            transformations, errors = staging_folder.get_transformation_list()
+                            staging_file.delete()
-                            staging_file.delete(preview_size=staging_folder.get_preview_size(), transformations=transformations)
                             messages.success(request, _(u'Staging file: %s, deleted successfully.') % staging_file.filename)
                         if document:
                             return HttpResponseRedirect(reverse('document_view_simple', args=[document.pk]))
@@ -280,15 +275,14 @@ def upload_interactive(request, source_type=None, source_id=None, document_pk=No
                             raise
                         messages.error(request, _(u'Unhandled exception: %s') % e)
             else:
-                form = StagingDocumentForm(cls=StagingFile,
+                form = StagingDocumentForm(document_type=document_type,
-                    document_type=document_type,
                     show_expand=(staging_folder.uncompress == SOURCE_UNCOMPRESS_CHOICE_ASK) and not document,
                     source=staging_folder,
                     instance=document
                 )
             try:
-                staging_filelist = StagingFile.get_all()
+                staging_filelist = list(staging_folder.get_files())
-            except Exception, e:
+            except Exception as e:
                 messages.error(request, e)
                 staging_filelist = []
             finally:
@@ -374,63 +368,20 @@ def get_form_filename(form):
     return filename
 
 
-def staging_file_preview(request, source_type, source_id, staging_file_id):
+def staging_file_delete(request, staging_folder_pk, filename):
     Permission.objects.check_permissions(request.user, [PERMISSION_DOCUMENT_CREATE, PERMISSION_DOCUMENT_NEW_VERSION])
-    staging_folder = get_object_or_404(StagingFolder, pk=source_id)
+    staging_folder = get_object_or_404(StagingFolder, pk=staging_folder_pk)
-    StagingFile = create_staging_file_class(request, staging_folder.folder_path)
-    transformations, errors = SourceTransformation.transformations.get_for_object_as_list(staging_folder)
 
-    output_file = StagingFile.get(staging_file_id).get_image(
+    staging_file = staging_folder.get_file(encoded_filename=filename)
-        size=staging_folder.get_preview_size(),
-        transformations=transformations
-    )
-    if errors and (request.user.is_staff or request.user.is_superuser):
-        for error in errors:
-            messages.warning(request, _(u'Staging file transformation error: %(error)s') % {
-                'error': error
-            })
-
-    return sendfile.sendfile(request, output_file)
-
-
-def staging_file_thumbnail(request, source_id, staging_file_id):
-    Permission.objects.check_permissions(request.user, [PERMISSION_DOCUMENT_CREATE, PERMISSION_DOCUMENT_NEW_VERSION])
-    staging_folder = get_object_or_404(StagingFolder, pk=source_id)
-    StagingFile = create_staging_file_class(request, staging_folder.folder_path, source=staging_folder)
-    transformations, errors = SourceTransformation.transformations.get_for_object_as_list(staging_folder)
-
-    output_file = StagingFile.get(staging_file_id).get_image(
-        size=THUMBNAIL_SIZE,
-        transformations=transformations
-    )
-    if errors and (request.user.is_staff or request.user.is_superuser):
-        for error in errors:
-            messages.warning(request, _(u'Staging file transformation error: %(error)s') % {
-                'error': error
-            })
-
-    return sendfile.sendfile(request, output_file)
-
-
-def staging_file_delete(request, source_type, source_id, staging_file_id):
-    Permission.objects.check_permissions(request.user, [PERMISSION_DOCUMENT_CREATE, PERMISSION_DOCUMENT_NEW_VERSION])
-    staging_folder = get_object_or_404(StagingFolder, pk=source_id)
-    StagingFile = create_staging_file_class(request, staging_folder.folder_path)
-
-    staging_file = StagingFile.get(staging_file_id)
     next = request.POST.get('next', request.GET.get('next', request.META.get('HTTP_REFERER', '/')))
     previous = request.POST.get('previous', request.GET.get('previous', request.META.get('HTTP_REFERER', '/')))
 
     if request.method == 'POST':
         try:
-            transformations, errors = SourceTransformation.transformations.get_for_object_as_list(staging_folder)
+            staging_file.delete()
-            staging_file.delete(
-                preview_size=staging_folder.get_preview_size(),
-                transformations=transformations
-            )
             messages.success(request, _(u'Staging file delete successfully.'))
-        except Exception, e:
+        except Exception as exception:
-            messages.error(request, _(u'Staging file delete error; %s.') % e)
+            messages.error(request, _(u'Staging file delete error; %s.') % exception)
         return HttpResponseRedirect(next)
 
     results = get_active_tab_links()

mayan/apps/sources/widgets.py

@@ -1,9 +1,14 @@
 from django import forms
-from django.utils.safestring import mark_safe
-from django.utils.encoding import force_unicode
 from django.conf import settings
-from django.utils.translation import ugettext_lazy as _
 from django.core.urlresolvers import reverse
+from django.utils.encoding import force_unicode
+from django.utils.html import strip_tags
+from django.utils.http import urlencode
+from django.utils.safestring import mark_safe
+from django.utils.translation import ugettext_lazy as _
+
+from converter.literals import DEFAULT_PAGE_NUMBER, DEFAULT_ROTATION, DEFAULT_ZOOM_LEVEL
+from documents.conf.settings import THUMBNAIL_SIZE, PREVIEW_SIZE
 
 
 class FamFamRadioFieldRenderer(forms.widgets.RadioFieldRenderer):
@@ -25,17 +30,55 @@ class FamFamRadioSelect(forms.widgets.RadioSelect):
     renderer = FamFamRadioFieldRenderer
 
 
-def staging_file_thumbnail(staging_file):
+def staging_file_thumbnail(staging_file, **kwargs):
-    try:
+    return staging_file_html_widget(staging_file, click_view='stagingfolderfile-image-view', **kwargs)
-        staging_file.get_valid_image()
-        template = u'<a class="fancybox-staging" href="%(url)s" title="%(filename)s" rel="staging")><img class="lazy-load" data-href="%(thumbnail)s" src="%(static_url)simages/ajax-loader.gif" alt="%(string)s" /><noscript><img src="%(thumbnail)s" alt="%(string)s" /></noscript></a>'
-    except:
-        template = u'<img class="lazy-load" data-href="%(thumbnail)s" src="%(static_url)simages/ajax-loader.gif" alt="%(string)s" /><noscript><img src="%(thumbnail)s" alt="%(string)s" /></noscript>'
 
-    return mark_safe(template % {
+
-        'url': reverse('staging_file_preview', args=[staging_file.source.source_type, staging_file.source.pk, staging_file.id]),
+def staging_file_html_widget(staging_file, click_view=None, page=DEFAULT_PAGE_NUMBER, zoom=DEFAULT_ZOOM_LEVEL, rotation=DEFAULT_ROTATION, gallery_name=None, fancybox_class='fancybox-staging', image_class='lazy-load', title=None, size=THUMBNAIL_SIZE, nolazyload=False):
-        'thumbnail': reverse('staging_file_thumbnail', args=[staging_file.source.pk, staging_file.id]),
+    result = []
-        'static_url': settings.STATIC_URL,
+
-        'string': _(u'thumbnail'),
+    alt_text = _(u'staging file page image')
-        'filename': staging_file.filename
+
-    })
+    query_dict = {
+        'page': page,
+        'zoom': zoom,
+        'rotation': rotation,
+        'size': size,
+    }
+
+    if gallery_name:
+        gallery_template = u'rel="%s"' % gallery_name
+    else:
+        gallery_template = u''
+
+    query_string = urlencode(query_dict)
+
+    preview_view = u'%s?%s' % (reverse('stagingfolderfile-image-view', args=[staging_file.staging_folder.pk, staging_file.encoded_filename]), query_string)
+
+    plain_template = []
+    plain_template.append(u'<img src="%s" alt="%s" />' % (preview_view, alt_text))
+
+    result.append(u'<div class="tc" id="staging_file-%s-%d">' % (staging_file.filename, page if page else DEFAULT_PAGE_NUMBER))
+
+    if title:
+        title_template = u'title="%s"' % strip_tags(title)
+    else:
+        title_template = u''
+
+    if click_view:
+        # TODO: fix this hack
+        query_dict['size'] = PREVIEW_SIZE
+        query_string = urlencode(query_dict)
+        result.append(u'<a %s class="%s" href="%s" %s>' % (gallery_template, fancybox_class, u'%s?%s' % (reverse(click_view, args=[staging_file.staging_folder.pk, staging_file.encoded_filename]), query_string), title_template))
+
+    if nolazyload:
+        result.append(u'<img style="border: 1px solid black;" src="%s" alt="%s" />' % (preview_view, alt_text))
+    else:
+        result.append(u'<img class="thin_border %s" data-original="%s" src="%simages/ajax-loader.gif" alt="%s" />' % (image_class, preview_view, settings.STATIC_URL, alt_text))
+        result.append(u'<noscript><img style="border: 1px solid black;" src="%s" alt="%s" /></noscript>' % (preview_view, alt_text))
+
+    if click_view:
+        result.append(u'</a>')
+    result.append(u'</div>')
+
+    return mark_safe(u''.join(result))

mayan/settings/includes/common.py

@@ -251,6 +251,12 @@ SERIALIZATION_MODULES = {
 SOUTH_MIGRATION_MODULES = {
     'taggit': 'taggit.south_migrations',
 }
+# ---------- Django REST framework -----------
+REST_FRAMEWORK = {
+    'PAGINATE_BY': 10,
+    'PAGINATE_BY_PARAM': 'page_size',
+    'MAX_PAGINATE_BY': 100,
+}
 
 try:
     from settings_local import *