Want to show your appreciation? Please to my charity.

Sunday, April 07, 2013

Integrate Java and GraphicsMagick – gm4java Performance

Introduction

In the last two posts of this series, we enhanced GraphicsMagick (GM hereafter) to support interactive or batch mode, and we developed the Java library, gm4java, to integrate with GM. Now it’s time to give it a run!

Application

This is the same web application that we tested the performance of sole im4java implementation. And we also use identical test environment and test setup for an apple to apple comparison. For the completeness, I’ll repeat the test application, environment and setup here.

The web application dynamically resize the source images that is 700x700 or less to a smaller dimension. An HTTP request come in with the individual image and the target size specified, the web application locates the source image and resize it to the target size, then streams the image back to the requester, typically a browser.

The application still use the same GM’s convert command,  “convert <source> –scale <width>x<height> <target>”, to scale down an image. The difference this time is that, instead of completely depending on im4java, we use gm4java to start 32 GM processes in interactive mode, they are pooled by gm4java to executed the convert commands coming in. That effectively eliminates the overhead of starting, and then shutting down, a new GM process for every request.

In order to avoid problems caused by any potential memory leak, each GM process was recycled after 1000 execution of commands. (Info: we have also tested with recycling disabled, that improved the performance by 1-2% and we also did not observe any memory leakage. But in the end we decided to keep the recycling enabled)

Test Environment

We also use exact same test environment that we tested the im4java integration. Again, repeated below:

Hardware

RAM: 32GB

CPU: 32 cores - 4 AMD Opteron(TM) Processor 6274 (8 cores)

Test Tool

JMeter is used to execute the test. The machine runs the JMeter has identical hardware as the test machine. There is high speed network connection between them and we have confirmed that there is no other factor being bottleneck.

Test Setup

We used 200K unique sample image and each image are being request twice, one converts to 260x420 and another converts to 114x166.

On thread group is used to run multiple threads in parallel. Each thread represent one concurrent user. We run the test multiple times, with different number of concurrent users. Each thread will request image one after another, there is no delay whatsoever.

We measure the throughput in terms of total number of images resized within a second by the server.

Test Result

Table below listed the total throughput under different load, side by side with the test result from our im4java implementation test.

Concurrent users Total Throughput (images/second)
im4java gm4java
50 278 Didn’t test
500 109 1447
5000 19 1490
20000 4 1452

The throughput is significantly higher comparing to the im4java implementation, and it stays high under high load as expected.

Disclaimer:this is NOT an comparison between im4java and gm4java, it is a comparison between the traditional integration mechanism im4java uses to integrate with GM, and the new interactive mechanism that is implemented by gm4java. gm4java does not compete with im4java, rather they complement each other.

Conclusion

The test result proves that using gm4java and GM’s new interactive/batch mode feature can provide a reliable and scalable mean to integrate Java and GM. We have successfully implemented our proposed solution, and concluded that is the solution we need.

Other Posts in the “Integrate Java and GraphicsMagick” Series

  1. Conception
  2. im4java Performance
  3. Interactive or Batch Mode
  4. gm4java
  5. gm4java Performance

10 comments:

Michael said...

Thats an very interesting article. Thats exactly what we need, becaus im4java ist to slow and jmagick ist to unreliable

I'm looking forward for further updates.

Kind Regards
Michael

Michi said...

Hello,

I need an RPM for Red Hat Enterprise Linux for GraphicsMagick.1.18.

If you know something, let me know. Otherwise I have to wait until it is official released.

Kind Regards
Michael

Kenneth Xu said...

@Michi, I guess you mean by 1.3.18 instead of 1.18. No I don't have binary RPM, I just downloaded the source and build it myself.

You can try to build binary RMP by following the instructions here: http://wiki.typo3.org/How_to_build_GraphicsMagick_RPM_package . I didn't try that myself so I don't know how accurate the instruction is. If you do try, I would appreciate if you can let me know the result.

Thanks!
Kenneth

Kenneth Xu said...

BTW, found one here: http://rpmfind.net//linux/RPM/fedora/devel/rawhide/x86_64/g/GraphicsMagick-1.3.18-1.fc19.i686.html

Again, didn't verify myself as I always compile from source.

Michi said...

Hello Kenneth Xu,

there are some new dependencies in the new GraphicsMagick.1.3.18. We could solve some dependencies, but not the glibc dependency, because it is to central. So we can't build and use it on RHEL 6. Now, we probaly have to wait until the next release of RHEL.

Finally, we have to use im4java and GraphicsMagick1.3.17 in our project.

But thank you for your help and this really good piece of software.

Best Regards
Michael

Kenneth Xu said...

@Michi, we have 1.3.8 installed on RHEL 6, see below. I'll get you more information.
$ cat /etc/redhat-release
Red Hat Enterprise Linux Server release 6.3 (Santiago)
gm
GraphicsMagick 1.3.18 2013-03-10 Q8 http://www.GraphicsMagick.org/
Copyright (C) 2002-2013 GraphicsMagick Group.
Additional copyrights and licenses apply to this software.
See http://www.GraphicsMagick.org/www/Copyright.html for details.
Usage: gm command [options ...]

Where commands include:
animate - animate a sequence of images
batch - issue multiple commands in interactive or batch mode
benchmark - benchmark one of the other commands
compare - compare two images
composite - composite images together
conjure - execute a Magick Scripting Language (MSL) XML script
convert - convert an image or sequence of images
display - display an image on a workstation running X
help - obtain usage message for named command
identify - describe an image or image sequence
import - capture an application or X server screen
mogrify - transform an image or sequence of images
montage - create a composite image (in a grid) from separate images
time - time one of the other commands
version - obtain release version

Kenneth Xu said...

@Michi and anyone need 1.3.18 on RHEL, you can build from source:

1. Make sure you have mercurial and gcc

sudo yum install mercurial
sudo yum install gcc

2. Build GraphicsMagick

hg clone http://hg.code.sf.net/p/graphicsmagick/code GM
cd GM
hg update -r GraphicsMagick-1_3_18
./configure --enable-shared
make

If everything goes well, you should be able to run your new built gm:

./utilities/gm

3. Install

sudo make install

4. Run gm from /usr/local:

$ /usr/local/bin/gm
GraphicsMagick 1.3.18 2013-03-10 Q8 http://www.GraphicsMagick.org/
Copyright (C) 2002-2013 GraphicsMagick Group.
Additional copyrights and licenses apply to this software.
See http://www.GraphicsMagick.org/www/Copyright.html for details.
Usage: /usr/local/bin/gm command [options ...]

Where commands include:
animate - animate a sequence of images
batch - issue multiple commands in interactive or batch mode
benchmark - benchmark one of the other commands
compare - compare two images
composite - composite images together
conjure - execute a Magick Scripting Language (MSL) XML script
convert - convert an image or sequence of images
display - display an image on a workstation running X
help - obtain usage message for named command
identify - describe an image or image sequence
import - capture an application or X server screen
mogrify - transform an image or sequence of images
montage - create a composite image (in a grid) from separate images
time - time one of the other commands
version - obtain release version

Kenneth Xu said...

And this is the version of glibc I use.
glibc x86_64 2.12-1.107.el6 ol6_latest 3.8 M
glibc-common x86_64 2.12-1.107.el6 ol6_latest 14 M

Michi said...

Hello Kenneth Xu,

thank you for your detailed information. You are right. Now I got it to work.

Thank you very much.

Best Regards
Michi

YouthOfUnity said...

I am using GraphicsMagick with gm4java , while using batch mode. But the performance is 4 times worst than command line execution. Plus gm also makes interim low , medium , high quality. How can i speed up the conversion ? Can i force GM not to supply high , medium low ? Is there a switch i can use ? the Virtual Machine has 8 CPUs, its an Amazon EC2 instance.
Thanks,
-Kamal.

Post a Comment