Archon DigitalLibrary_Files Table Size Problem

For issues with installing and configuring the Archon software to begin running on your webserver.

Moderators: paulnsorensen, kylefox2, prom, rishel

Archon DigitalLibrary_Files Table Size Problem

Postby pastori » Fri Mar 09, 2012 4:46 pm

HI FOLKS:

Is there a recommended maximum size for the database content data field(s)? Our MySQL server is choking on the size of the data in the Archon DigitalLibrary_Files Table. It crashes the server which kills access to all the other application schemas. We are running an Apache webserver on a SUN Solaris server to display the Archon Data and MySQL is running on a separate server and is using 12GB of memory.

The Archon DigitalLibrary_Files Table has 3,160 records and a size of 7.6GB. Most of the data is PDF file content which is stored in the following three fields: FileContents, FilePreviewLong and FilePreviewShort fields of each record. The largest single PDF content size is 10,640,203 bytes .

The MySQL management program tools that I use can NOT read the records in this table completely because the extremely
large records overload memory and cause the program to abort.

The following shows the MySQL server parameters that are set:
skip-external-locking
key_buffer_size = 96M
sort_buffer_size = 2M
read_buffer_size = 2M
read_rnd_buffer_size = 8M
join_buffer_size = 96M
table_cache = 2048
general-log = 'YES'
slow-query-log = 'YES'
myisam_use_mmap = 1
myisam_sort_buffer_size = 96M
thread_stack = 256K
thread_cache_size = 64
max_allowed_packet = 16M
max_connections = 150
##max_connect_errors=100
wait_timeout = 100
max_heap_table_size = 128M
tmp_table_size = 128M
query_cache_size= 48M
query_cache_limit = 4M
# Try number of CPU's*2 for thread_concurrency
thread_concurrency = 16.

Any help in solving this performance problem is greatly appreciated. Thanks in advance.
pastori
 
Posts: 1
Joined: Fri Mar 09, 2012 4:09 pm

Re: Archon DigitalLibrary_Files Table Size Problem

Postby paulnsorensen » Mon Mar 12, 2012 2:57 am

The real answer is that the files shouldn't be in the database to begin with. I know this doesn't provide a solution, but I think until there is a way for Archon to solely store the files in the filesystem, or until ArchivesSpace is released, the best thing you can do is export the current state of your PDFs and save them elsewhere, and then compress the largest files and reupload them in the mean time.
paulnsorensen
Site Admin
 
Posts: 631
Joined: Mon Mar 31, 2008 8:55 am


Return to Installation/Configuration Issues

Who is online

Users browsing this forum: Yahoo [Bot] and 2 guests

cron