Merge branch 'hotfix/v0.12.1' into development

Conflicts:
	apps/converter/conf/settings.py
	apps/documents/__init__.py
	apps/main/__init__.py
	apps/metadata/__init__.py
	apps/navigation/templatetags/navigation_tags.py
	apps/ocr/__init__.py
	apps/ocr/conf/settings.py
	docs/intro/installation.rst
	docs/releases/index.rst
	requirements/production.txt
This commit is contained in:
Roberto Rosario
2012-06-03 22:50:33 -04:00
91 changed files with 2080 additions and 574 deletions
+2 -2
View File
@@ -52,10 +52,10 @@ copyright = u'2011, Roberto Rosario'
# built documents.
#
# The short X.Y version.
version = '0.12'
version = '0.12.1'
# The full version, including alpha/beta/rc tags.
release = '0.12'
release = '0.12.1'
# The language for content autogenerated by Sphinx. Refer to documentation
# for a list of supported languages.
+10 -6
View File
@@ -15,7 +15,7 @@ Bug fixes
---------
* Aziz M. Bookwala (https://github.com/azizmb)
* IHLeanne (https://github.com/IHLeanne)
* Сергей Глита [Sergey Glita] (s.v.glita@gmail.com)
* Sergey Glita (s.v.glita@gmail.com)
* Meurig Freeman (https://github.com/meurig)
* David Herring (https://github.com/abadger1406)
@@ -27,24 +27,27 @@ Bug reports
* Joost Cassee (joost@cassee.net, https://github.com/jcassee)
* Brian Huxley
* dAnjou (https://github.com/dAnjou)
* Сергей Глита [Sergey Glita] (s.v.glita@gmail.com)
* Sergey Glita (s.v.glita@gmail.com)
* IHLeanne (https://github.com/IHLeanne)
* valterwill (https://github.com/valterwill)
* David Herring (https://github.com/abadger1406)
* Alexandru Kiss (usurel@gmail.com)
Patches
-------
* Meurig Freeman (https://github.com/meurig)
* Сергей Глита [Sergey Glita] (s.v.glita@gmail.com)
* Sergey Glita (s.v.glita@gmail.com)
* Brian E (brian@realize.org)
Suggestions
-----------
* Cezar Jenkins (https://twitter.com/#!/emperorcezar)
* Сергей Глита [Sergey Glita] (s.v.glita@gmail.com)
* Sergey Glita (s.v.glita@gmail.com)
* Barry Rowlingson (http://geospaced.blogspot.com)
* Gour (https://github.com/gour)
* Alexandru Kiss (usurel@gmail.com)
Translations
@@ -56,7 +59,7 @@ Translations
* Russian
- Сергей Глита [Sergey Glita] (s.v.glita@gmail.com)
- Sergey Glita (s.v.glita@gmail.com)
* Italian
@@ -69,9 +72,10 @@ Translations
Remote access for debugging
---------------------------
* Сергей Глита [Sergey Glita] (s.v.glita@gmail.com)
* Sergey Glita (s.v.glita@gmail.com)
* David Herring (https://github.com/abadger1406)
* Michael Terretta (terretta@gmail.com)
* Alexandru Kiss (usurel@gmail.com)
Monetary donations
+1
View File
@@ -23,6 +23,7 @@ On the Web
* Website: http://www.mayan-edms.com
* Source: http://github.com/rosarior/mayan
* Video: http://bit.ly/Mayan-Intro
* Mailing list: http://groups.google.com/group/mayan-edms/
Looking for specific information? Try the :doc:`detailed table of contents <contents>` otherwise below are the different part of the documentation.
+182 -13
View File
@@ -2,14 +2,90 @@
Installation
============
Local or managed server
-----------------------
Automatic install using included fabfile
----------------------------------------
A Fabric_ file is included to help users not very familiar with Ubuntu_,
Python_ and Django_ install **Mayan EDMS**, or for system administrators
looking to automate the install whether in a local or remote system.
To bootstrap **Mayan EDMS** via the fabfile without having to clone the
entire repository, run the following command, replacing the part that
reads: <Your MySQL root password> with your current MySQL root password
or the MySQL root password you plan to assign to it, during the MySQL
installation when executing the fabfile.
* Debian or Ubuntu::
$ cd /tmp && sudo apt-get install -y fabric wget tar gzip && wget https://raw.github.com/rosarior/mayan/contrib/fabfile.tar.gz -O - | tar -xvzf - && echo "database_manager_admin_password=<Your MySQL root password>" > ~/.fabricrc && fab -H localhost install
* Fedora::
$ cd /tmp && sudo yum install -y fabric wget tar gzip && wget https://raw.github.com/rosarior/mayan/contrib/fabfile.tar.gz -O - | tar -xvzf - && echo "database_manager_admin_password=<Your MySQL root password>" > ~/.fabricrc && fab -H localhost install
The ``fabfile`` also supports deploying to more than one server whether
the they same configuration or not, via a ``server_config.json`` file.
The fabfile also supports YAML server configuration
files but requires installing ``pyyaml`` via ``pip``, ``python-yaml``
via ``apt`` (Debian_, Ubuntu_) or ``PyYAML`` via ``yum`` (Fedora_)
A sample ``server_config.yaml`` would be as follows::
debian_server:
host: 192.168.1.1
database_manager_admin_password: my_debian_mysql_pa$$word
drop_database: True
os: debian
ubuntu_server:
host: 192.168.1.2
database_manager_admin_password: my_ubuntu_mysql_pa$$word
drop_database: True
os: ubuntu
database_username: mayan_release_12
database_name: mayan_release_12
all_servers: [debian_server, ubuntu_server]
To install **Mayan EDMS** to these servers the ``fabfile`` should be called in the following manner::
$ fab servers:debian_server install
or::
$ fab servers:ubuntu_server,debian_server install
or::
$ fab servers:all_servers install
Configuration options for the ``fabfile``:
* ``host`` Hostname or address of the host to which to install **Mayan EDMS**. Required setting.
* ``os`` Operating system of the target host. Options: ubuntu, debian, fedora. default: ubuntu
* ``install_path`` Installation path. Default: /usr/share
* ``virtualenv_name`` Name of the directory that will house the virtualenv under the ``install_path``. Default: mayan
* ``repository_name`` Name of the directory where the repository will be clone under the ``virtualenv_name`` directory. Default: mayan
* ``database_manager`` Database manager to use. Options: mysql. Default: mysql
* ``database_username`` Username that will be created and used to access the database. Default: mayan
* ``database_password`` Password of the database user account. Default: random autogenerated password
* ``database_host`` Address of the database manager host. Default: 127.0.0.1
* ``drop_database`` Weather or not to drop the database when uninstalling **Mayan EDMS** via the fabfile. Default: False
* ``database_manager_admin_password`` Administrator password of the database manager, only used to create the **Mayan EDMS** database and user. Required setting.
* ``database_name`` Name of the **Mayan EDMS** database. Default: mayan
* ``webserver`` Web server that will be install and configure to server **Mayan EDMS**. Options: apache. Default: apache
Only the options: ``host``, ``os`` and ``database_manager_admin_password`` should be required for most installations.
Local or managed Ubuntu or Debian server
----------------------------------------
**Mayan EDMS** should be deployed_ like any other Django_ project and preferably using virtualenv_.
If using a Debian_ or Ubuntu_ based Linux distribution getting the executable requirements is as easy as::
$ apt-get install tesseract-ocr unpaper python-virtualenv ghostscript -y
$ sudo apt-get install python-dev gcc tesseract-ocr unpaper python-virtualenv ghostscript libjpeg-dev libpng-dev poppler-utils -y
To initialize a ``virtualenv`` to deploy the project do::
@@ -34,19 +110,35 @@ To install the python dependencies ``easy_install`` can be used, however for eas
Create the database that will hold the data. Install any corresponding python database drivers. Update the settings.py file with you database settings.
If using the ``MySQL`` database manager, use the following commands::
$ apt-get install python-dev libmysqlclient-dev gcc -y
$ sudo apt-get install libmysqlclient-dev -y
$ pip install MySQL-python
If using PostgreSQL, enter the following::
If using ``PostgreSQL``, enter the following::
$ apt-get install python-dev libpq-dev gcc-y
$ pip install pip install psycopg2
$ sudo apt-get install libpq-dev -y
$ pip install psycopg2
Populate the database with the project's schema doing::
$ ./manage.py syncdb --migrate
Collect the static files of the project into the ``static`` folder for serving via a webserver::
To test your installation, create a file called settings_local.py with the following content::
DEBUG=True
DEVELOPMENT=True
Execute Djangos development server using the ``runserver`` command to launch a local instance of Mayan EDMS::
$ ./manager.py runserver
Point your browser to http://127:0.0.1:8000, if everything was installed
correctly you should see the login screen. After making sure everything
is running correctly, stop the runserver command and delete the settings_local.py.
Deploy **Mayan EDMS** using the webserver of your preference. If your are
using Apache_, a sample site file is included under the contrib directory.
Before finally deploying to your favorite webserver don't forget to collect the
static files of the project into the ``static`` folder for serving via a webserver::
$ ./manage.py collectstatic
@@ -59,6 +151,85 @@ Execute Django's runserver command to launch a local instance of **Mayan EDMS**
After making sure everything is running correctly, stop the runserver command, delete the settings_local.py and deploy **Mayan EDMS** using the webserver of your preference. If your are using Apache_, a sample site file is included under the contrib directory.
Local or managed Fedora server
------------------------------
**Mayan EDMS** should be deployed_ like any other Django_ project and preferably using virtualenv_.
If using a Fedora_ based Linux distribution getting the executable requirements is as easy as::
$ sudo yum install -y git gcc tesseract unpaper python-virtualenv ghostscript libjpeg-turbo-devel libpng-devel poppler-util python-devel
To initialize a ``virtualenv`` to deploy the project do::
$ virtualenv --no-site-packages mayan
Download_ and decompress the latest version of **Mayan EDMS**::
$ cd mayan
$ tar -xvzf mayan.tar.gz
Or clone the latest development version straight from github::
$ cd mayan
$ git clone git://github.com/rosarior/mayan.git
To install the python dependencies ``easy_install`` can be used, however for easier retrieval a production dependencies file is included, to use it execute::
$ cd mayan
$ source ../bin/activate
$ pip install -r requirements/production.txt
Create the database that will hold the data. Install any corresponding python database drivers. Update the settings.py file with you database settings.
If using the ``MySQL`` database manager, use the following commands::
$ sudo yum install -y mysql-devel
$ pip install MySQL-python
If using ``PostgreSQL``, enter the following::
$ sudo yum install -y postgresql-devel
$ pip install psycopg2
Populate the database with the project's schema doing::
$ ./manage.py syncdb --migrate
To test your installation, create a file called settings_local.py with the following content::
DEBUG=True
DEVELOPMENT=True
Execute Djangos development server using the ``runserver`` command to launch a local instance of Mayan EDMS::
$ ./manager.py runserver
Point your browser to http://127:0.0.1:8000, if everything was installed
correctly you should see the login screen. After making sure everything
is running correctly, stop the runserver command and delete the settings_local.py.
Deploy **Mayan EDMS** using the webserver of your preference. If your are
using Apache_, a sample site file is included under the contrib directory.
Before finally deploying to your favorite webserver don't forget to collect the
static files of the project into the ``static`` folder for serving via a webserver::
$ ./manage.py collectstatic
Cloud install
-------------
SaaS provied Appsembler_ has started providing a "1-click install" cloud
offering of **Mayan EDMS**. Go to their website and click on apps to start
your trial period of **Mayan EDMS** on the cloud.
DjangoZoom
----------
For instructions on how to deploy **Mayan EDMS** on DjangoZoom, watch the screencast:
"Deploying Mayan EDMS on DjangoZoom.net" available on Youtube_
Webfaction
----------
@@ -168,11 +339,6 @@ To install **Mayan EDMS** on Webfaction_, follow these steps:
apache2/bin/restart
DjangoZoom
----------
For instructions on how to deploy **Mayan EDMS** on DjangoZoom, watch the screencast:
"Deploying Mayan EDMS on DjangoZoom.net" available on Youtube_
.. _`vendor lock-in`: https://secure.wikimedia.org/wikipedia/en/wiki/Vendor_lock-in
@@ -193,3 +359,6 @@ For instructions on how to deploy **Mayan EDMS** on DjangoZoom, watch the screen
.. _deployed: https://docs.djangoproject.com/en/1.3/howto/deployment/
.. _virtualenv: http://www.virtualenv.org/en/latest/index.html
.. _`Reducing mod_wsgi Memory Consumption`: http://docs.webfaction.com/software/mod-wsgi.html#mod-wsgi-reducing-memory-consumption
.. _Fedora: http://fedoraproject.org/
.. _Fabric: http://docs.fabfile.org/
.. _Appsembler: http://appsembler.com/
+1 -1
View File
@@ -39,7 +39,7 @@ Version 0.10
* Document image serving response now specifies a MIME type for increased
browser compatibility.
* Small change in the scheduler that increases stability.
* Russian translation updates (Сергей Глита [Sergey Glita])
* Russian translation updates Sergey Glita
* Improved and generalized the OCR queue locking mechanism, this should
eliminate any posibility of race conditions between Mayan EDMS OCR nodes.
* Added support for signals to the OCR queue, this results in instant OCR
+116
View File
@@ -0,0 +1,116 @@
================================
Mayan EDMS v0.12.1 release notes
================================
*May 2012*
This is the first maintenance release of the 0.12 series.
Overview
========
While bug fixes and minor feature were the focus for this release, some
bigger changes were included because of their importance. The parsing of
documents saw a complete rewrite being now class based and allows for more
than one parser per mimetype with sequencial fallback. This provides the
best text extraction on deployments where users have control over the
installation and basic extraction when deploying on the cloud or other
environments where users don't have the ability to install OS level
binaries.
What's new in Mayan EDMS v0.12.1
================================
Fabric file (fabfile)
~~~~~~~~~~~~~~~~~~~~~
A Fabric file is included to help users not very familiar with Ubuntu,
Python and Django install **Mayan EDMS**, or for system administrators
looking to automate the install whether in local or remote systems.
At the moment the fabfile will install **Mayan EDMS** in the same configurations
listed in this documentation, that is: (Ubuntu/Debian/Fedora) + virtualenv + Apache + MySQL.
Feel free to submit your configuration settings and files for different databases,
webserver or Linux distribution. More configurations will be added to
the fabfile as more are tested.
Documentation update
~~~~~~~~~~~~~~~~~~~~
The installation instructions were updated to include the installation of
the libpng-dev and libjpeg-dev libraries as well as the installation of
the poppler-utils package. An additional step to help users test their
new installation of **Mayan EDMS** was also added.
Translations
~~~~~~~~~~~~
The Italian translation has been synchronized with the source files at
Transifex and finished to %100 completion.
Usability improvements
~~~~~~~~~~~~~~~~~~~~~~
The index instance view now feature the same multi document action
buttons (Submit to OCR, delete, download, etc) as the mail and recent
document views.
Better office document conversion
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
A new method of converting office documents has been implemented, this
new method doesn't require the use of the command line utility ``UNOCONV``.
If this new method proves to work better than previous solutions the use
of ``UNOCONV`` may be deprecated in the future. The conversion method
adds just one new configuration option: :setting:`CONVERTER_LIBREOFFICE_PATH`
which defaults to '/usr/bin/libreoffice'.
Better PDF text parsing
~~~~~~~~~~~~~~~~~~~~~~~
Brian E. submitted a patch to use the Poppler package pdftotext utility to
extract text from PDF files. This is now the default method Mayan EDMS
will execute to try to extract text from a PDF and failing that will
fallback to the previous method. This change add a new configuration
option: :setting:`OCR_PDFTOTEXT_PATH` to specify the location of the ``pdftotext``
executable, it defaults to '/usr/bin/pdftotext'. Be sure to install the
``poppler-utils`` os package to take advantage of this new parser.
Changed defaults
~~~~~~~~~~~~~~~~
The OCR queue is now active by default when first created during the
``syncdb`` phase and the :setting:`OCR_AUTOMATIC_OCR` option now defaults
to ``True``. These two changes are made to reduce the steps required for
new users to start enjoying the benefits of automatic text extraction from
uploaded documents without having to read the documentation and have a more
functional default install.
Upgrading from a previous version
=================================
Start off by adding the new requirements::
$ pip install -r requirements/production.txt
Migrate existing database schema with::
$ ./manage.py migrate documents
Install the ``poppler-utils`` package:
* Ubuntu, Debian::
$ apt-get install -y poppler-utils
* Fedora::
$ yum install -y poppler-utils
The upgrade procedure is now complete.
Backward incompatible changes
=============================
* None
Bugs fixed
==========
* Issue #25 "Office document conversion error"
Stuff removed
=============
* None
+2 -2
View File
@@ -1,7 +1,7 @@
Version 0.9.1
-------------
* Added handling percent encoded unicode query strings in search URL,
thanks to (Сергей Глита [Sergei Glita]) for reporting.
thanks to Sergei Glita for reporting.
* Added a FAQ explaing how to fix MySQL collation related error when
doing searches also thanks to (Сергей Глита [Sergei Glita]) for
doing searches also thanks to Sergei Glita for
reporting this one.
+2 -2
View File
@@ -16,11 +16,11 @@ Version 0.9
* Changed to Semantic Versioning (http://semver.org/), with
recommendations 7, 8 and 9 causing the most effect in the versioning number.
* Added Russian locale post OCR cleanup backend (Сергей Глита [Sergei Glita])
* Added Russian locale post OCR cleanup backend Sergei Glita
* Reduced severity of the messages displayed when no OCR cleanup backend
is found for a language
* Complete Portuguese translation (Emerson Soares and Renata Oliveira)
* Complete Russian translation (Сергей Глита [Sergei Glita])
* Complete Russian translation Sergei Glita)
* Added animate.css to use CSS to animate flash messages with better
fallback on non JS browsers
* The admin and sentry links are no longer hard-coded (Meurig Freeman)
+2 -1
View File
@@ -9,7 +9,7 @@ changes made in that version.
For those upgrading to a new version of **Mayan EDMS**, you will need to check
all the backwards-incompatible changes and deprecated features for
each 'final' release from the one after your current **Mayan EDMS** version,
up to and including the new version.
up to and including the latest version.
Latest version (0.13)
---------------------
@@ -17,6 +17,7 @@ Latest version (0.13)
:maxdepth: 1
0.13
0.12.1
0.12
Historic changelogs
+24 -3
View File
@@ -200,7 +200,7 @@ fine tune it's functionality as explained in the `GraphicsMagick documentation`_
Default: ``/usr/bin/unoconv``
Path to the unoconv program used to call LibreOffice for office document convertion.
Path to the unoconv program used to call LibreOffice for office document conversion.
.. setting:: CONVERTER_UNOCONV_USE_PIPE
@@ -211,8 +211,19 @@ Path to the unoconv program used to call LibreOffice for office document convert
Default: ``True``
Use alternate method of connection to LibreOffice using a pipe, it is slower but less prone to segmentation faults.
.. setting:: CONVERTER_LIBREOFFICE_PATH
**CONVERTER_LIBREOFFICE_PATH**
Default: ``/usr/bin/libreoffice``
Path to the libreoffice binary used to call LibreOffice for office document conversion.
Linking
=======
@@ -341,7 +352,7 @@ Maximum amount of concurrent document OCRs a node can perform.
**OCR_AUTOMATIC_OCR**
Default: ``False``
Default: ``True``
Automatically queue newly created documents or newly uploaded versions
of existing documents for OCR.
@@ -362,7 +373,17 @@ Default: ``/usr/bin/unpaper``
File path to the ``unpaper`` executable, used to clean up images before
doing OCR.
.. setting:: OCR_PDFTOTEXT_PATH
**OCR_PDFTOTEXT_PATH**
Default: ``/usr/bin/pdftotext``
File path to ``poppler's`` ``pdftotext`` program used to extract text
from PDF files.
Metadata
========