Wednesday, May 28, 2014

"How To Make Kindle Comics" Update 2.0

A newly revised and updated "2.0" edition of "How To Make Kindle Comics & Children's Books" has just been published, with additional content and revisions to include the recent changes made by Amazon to the allowed image file size limits in Kindle fixed layout ebooks for HD devices, as well as other details uncovered during file testing since the first edition was released.

I have also taken the opportunity to make some editorial changes, textual polishing, and adjustments to the relevant charts and sections affected by these recent changes. Much of this has been documented on this blog, but it is now all housed together in the published guide.

In addition, an ePub edition is now included in the downloaded zip archive for no extra charge when purchased through my website (along with an updated PDF). The ePub and PDF are not available if bought through Amazon! However, the Kindle edition sold there has also been updated, so you should see an "update available" button for this title in the "Manage Your Content" page at Amazon (to the right of the title).

When you click this button you'll receive the following info message box:

If you've made any notes or highlights in the book, these will be lost, since the content on the pages doesn't actually match up with the previous version anymore. In addition to a substantial number of revisions, additions, and alterations, the book is now two pages longer. This is why Amazon don't automatically push updated versions of a title to your device.

For those of you who have already purchased the book directly from my site, you can download the updated files via the link you received when purchased. If you no longer have that link, drop a line via the Contact page and I'll resend it. Be sure to include your full name and the email address you used when you bought the book.

The sample pdf download has also been updated, so for those who have not yet acquired a copy, you can click the "Read Sample" button on the product page (or just click that link) to read a 40+ page excerpt of the book.

Sunday, May 25, 2014

Kindlegen "-dont_append_source" Option

I discovered this week that there is a command line option for Kindlegen that keeps the source file zip from being added to the compiled mobi archive. It was hidden in the Kindle Publishing Guidelines under the "Building Dictionaries" section (7.5), appearing first in Version 2014.1 back in January, and was not specifically listed in the Revision History, where it simply said "Dictionary Overview (and all subsections of same)." I went through that section, noting all the changes but this one. Go figure.

At any rate, if you add this option to your kindlegen command it will bypass the addition of the source file archive to your output file:
Kindlegen will output this message as the first line in the conversion log:
Info:I9018:option: -donotaddsource: Source files will not be added
I tested a fixed layout file to be sure it worked, and what I found was that a 19.6 Mb source epub which normally converts to a 44.9 Mb file, resulted in a 25.2 Mb mobi when appending this conversion option. Extracting the contents using KindleUnpack showed that the source zip file was indeed absent.

So for those who have been using KindleStrip to remove the bulky source files from your files, you can now just add this option during conversion instead. KindleStrip is no longer necessary.

Bear in mind, however, that with the larger image file size allowance, if you're including higher resolution images for HD devices that are over the standard definition limits (i.e. 127/256/800 kb), these source files may be required by Amazon in order to send them to the end user's HD device.

UPDATE: 5-27-14

In fact, the archive is not required in order for the HD images to be delivered, as these are stored in a new HD Container section of the converted mobi file. So you can safely remove the zip archive (or keep it from being added in the first place) when distributing the files outside of Amazon. Of course, there's no real reason to remove them if uploading to KDP, unless your converted file exceeds 50 Mb in size.

Saturday, May 24, 2014

Kindle Image Size Test Results

I ran some further tests this morning to determine the effects of various image file sizes in different Kindle formats, on different Kindle devices. I did this in order to determine where the cutoff was for acceptable response, since the first test I did last week resulted in very sluggish behavior on the HD 8.9 device, which is the only one I tested then. I had also only tested a fixed layout file with the comic book-type on the actual device, so I did not know if these results applied to children's books or reflowable files as well.

For this test I created a jpeg image using a color gradient and pattern overlay in Photoshop to produce a range of sharp and soft edges across the spectrum, as well as an area of sharp black and white lines. Initially I made a file that was 4800 x 3000 pixels in size, since this is the largest image size recommended by Amazon in the Kindle Publishing Guidelines, in Section 5.2, where it deals with Zoom Factors.

However, I could not create a file at this size with this much color information at 100% quality that was under the 5Mb image limit. In fact, the file came in at 7.97 Mb, and even though it exceeds the limit by nearly 3 Mb, Kindlegen accepted it and did not abort. The resulting quality, of course, was horrible, even on the HD 8.9, and the image was clearly compressed to fit the limit. Moreover, after extracting the contents of the file, I discovered that Kindlegen will, in fact, actually reduce the dimensions of an image this size, even in a file with the comic book-type, contrary to my previous beliefs (though, of course, I had never tested an image file anywhere near this big, since it was not previously allowed). Therefore, apparently Kindlegen will no longer abort during conversion due to image file size, even when the image is bigger than the new limit.

I next created a new jpeg image file that was 3800 x 2376 in size and came it right at 5Mb with no compression applied (that is, a 100% quality factor in Photoshop). Images at this resolution did not have their dimensions altered by Kindlegen. From this image I then created a series of versions, each with successively more compression applied, right down to 10% "low" quality. These were the file size results:

100% - 5 Mb
95% - 4.30 Mb
90% - 3.8 Mb
80% - 2.9 Mb "Very High Quality"
70% - 2.4 Mb
60% - 2.06 Mb "High Quality"
50% - 1.39 Mb
30% - 1.04 Mb "Medium Quality"
10% - 807 Kb "Low Quality"

With these I produced a 10-page ebook (including a cover image using the 100% file, since this image is compressed differently than internal image files), with each page containing an image of greater compression than the one preceding it. I then created both a fixed layout "comic" and "childrens" book, as well as a reflowable file from this source archive. Total file sizes were 46.1 Mb, 44.7 Mb, and 40.3 Mb respectively. I tested each of these on the HD 8.9, HD 7, and the Kindle Fire 2. I did not bother with the eInk screens, whose image clarity is poor at best.

I should note here that Photoshop handled the compression of this image rather admirably, with "reasonably" acceptable results right down through the lower range, with minor amounts of blurring occurring from around 80%, and visible artifacting beginning at 60%, becoming plainly obvious at 30%, but never gravely intolerable (and I'm talking about a fairly small amount overall, visible only with close scrutiny). The quality of the images on the HD devices was, therefore, equally decent, with no overly glaring errors or corruption, even at the lowest level of 10%. But overall the images were vivid and pristine on the HD displays throughout upper quality levels, and quite good across the whole range. They would, of course, be even better with images of smaller dimensions, but I wanted to test the extreme limits of file size.

Ironically, on the standard resolution Fire these aberrations were less noticeable, due to an overall blurry appearance of the images at all levels, since on these devices the Kindlegen-compressed images are displayed rather than the full size hi-rez images, and thus they are all highly compressed (i.e. they all sucked more or less equally). The reflowable file was, of course, much less tolerable than the other two, with all images suffering from severe fuzziness and lack of crisp details. This is due to the fact that Kindlegen was forced to reduce the image dimensions down to around 575 x 920 (from 3800 x 2376) in order to achieve the 127 kb limit for reflowable files, requiring even the "low resolution" Fire screens to add back in additional pixel information that had been removed during conversion! Conversely, on the HD displays the reflowable file images were crisp and pristine, proving that the original resolution images were being displayed.

But this is nothing the average reader would likely even notice, aside from a general sense of mediocre presentation, as opposed to the sense of pristine clarity that comes across in hi-rez files on the HD screens. Without both devices sitting side by side for comparison the difference in detail would be far less obvious.

Of course, image quality was only part of what I was after here. This test was undertaken as much for the purpose of determining the effects of image file size on page load time. In my initial test last week the high resolution images I used caused the HD 8.9 to bog down to a crawl, but I did not know if this was due to the image size, dimensions, book-type, device, or combination of them all. So with the files I made today I loaded each one onto the devices mentioned above and swiped through the pages methodically, counting the seconds it took for each page to load, and then repeating it at least three times each, for each book-type on each device. Tedious, to say the least.

Interestingly, what I found was that in every instance - with the single exception of the comic book-type on the HD 8.9 - the results were more or less the same. The chart below shows the average load time in seconds, with the exception of the HD 8.9 comics in its own column:

File Size
HD 8.9 Comics
All Other
100% - 5Mb
95% - 4.3 Mb
90% - 3.8 Mb
80% - 2.9 Mb
70% - 2.4 Mb
60% - 2.06 Mb
50% - 1.39 Mb
30% - 1.04 Mb
10% - 807 Kb

As you can see, the load time more than doubles for the HD 8.9 device with the comic book-type, ranging from 3-5 seconds rather than the 1-2 second load time of all other instances. Moreover, response was buggy and caused the book to close unintentionally on numerous occasions. This is possibly due to the additional system feature of Virtual Panels that has to load with each page in comics, although this also occurs on the HD7 and Fire, just not at that resolution. But it is certainly not due to the image size itself, since this does not occur on that device with children's books or reflowable files that include the exact same images.

At any rate, from the chart you can see that in most cases the image load time is in the 1 second range with images of 2 Mb or less, while images from 2.4 to 3.8 Mb take slightly longer, and 4-5 Mb images requiring twice as long to load. For comics on the HD 8.9 there was a larger degree of variance throughout the tests, with more inconsistency in load times, but in general pages that contained an image larger than 2.4 Mb took on average a full 5 seconds to load - an excruciatingly long wait, during which time the device simply does nothing at all (on the HD 7, conversely, the page turns white and a circular "loading" icon appears, so that at least you know something is happening). The average dropped to an only slightly less annoying 4 seconds for 1-2 meg files, and only reached a 3 second best for the lowest quality image - one that only just exceeds the standard comic file size allowance of 800 kb!

Bear in mind, of course, that these are still also very large files in terms of image dimensions, so that regardless of the file size there is still an enormous amount of pixel data for the display to deal with. Moreover, these results will improve in time as processor speeds and onboard memory increase. But for now, as I've said on many occasions before, we have to deal with what we've got. My recommendation is to use much smaller dimensions (say, 2880 x 1800 max, which is 150% of the HD 8.9" resolution), and shoot for around 1.5 to 2 Mb for your largest image file size (and smaller if you can get away with it without compromising quality, bearing in mind overall file size and the subsequent delivery fees). And this all obviously only applies to images with a high degree of pixel information in them, such as complex maps and highly detailed artwork. Use your best judgement, of course, but be aware that a much higher degree of image quality is now possible on the Kindle than it ever has been before. Take advantage of it if you can. I, for one, look forward to seeing some truly beautiful artwork radiating from my HD Kindle screens.

Saturday, May 17, 2014

Kindle Image File Size Increase Confirmed

After a lengthy wait, I finally received confirmation from Amazon this week regarding the increase in allowed image files sizes for Kindle ebooks listed in the newest version of the Kindle Publishing Guidelines, as discussed in recent posts.

While the image size limit is now stated to be 5 Mb per image, rather than the previous 127 kb for reflowable files and 256 or 800 kb for children's and comic fixed layouts respectively, there has been no official announcement from Amazon on the matter. Moreover, there has been no release of an update to Kindlegen that might incorporate this change, and files converted via Kindlegen or Previewer still compress the images to the previous levels, as revealed by extracting the contents using KindleUnpack.

How, then, is it possible for the higher 5 Mb image limit listed in the Guidelines to be true? Is this just a future upgrade waiting to be made? A simple typographic error? Or is there some unwritten information hidden between the lines?

In an email response to my query on the matter received this week, an Amazon rep said:
From Version 2.9 of Kindlegen, images up to 5MB are retained for High Definition devices. For Standard Definition devices, however, the images will be compressed.
The answer, then, is different images for different devices! In fact, Amazon has apparently been doing this for some time already without telling anyone. Since the converted mobi file contains both the compressed images and an untouched source file zip archive, Amazon is able to send alternate versions of each ebook to different devices, depending on their resolution.

Of course, I wanted to know for certain if, and how, this worked in practice, so I did some tests this morning to determine if it was, in fact, the case. To do this I created a fixed layout, comic book-type test file with 6 pages, each containing just one full page image, ranging in file size from 1.8 to 4.94 megabytes. This created a source file of 22.2 Mb in size, which Previewer converted into a mobi file of 48.6 megs. I also uploaded the source file to KDP, which resulted in a download preview file of an only slightly smaller 45.5 Mb. In addition, I tested variations with the children's book-type, and as a reflowable file, all of which resulted in visible artifacting in the compressed images when extracted. However, when these same files were side-loaded to my HD 8.9" Kindle, every image looked crystal clear, showing that the reading system was accessing not the compressed images, but the high definition source files instead!

Here is a screen-cap comparison of the high quality image file being rendered on the HD 8.9 (left) and the compression employed on the same image for the standard definition file (on the right):

The HD device is clearly accessing the uncompressed image from the source archive file, rather than the highly compressed version found in the mobi8 image folder. Now, bear in mind that this is a 127 kb image that has been compressed down from one nearly 5 mb in size, so an incredible amount of compression has occurred. Moreover, Kindlegen will shrink the actual resolution of the image in reflowable files and fixed layouts with the children's book-type, although not in ones with the comic book-type, so this comparison is between the two extremes and probably not a common occurrence. But you should know that this can happen, and is, in fact, happening to your ebooks already.

One more important note must be mentioned here with regard to including very high resolution images, which is that it severely bogged down the device response, causing delays in page turns, sluggish response to scrolling and zooming, and in general behaving in a most unpleasant way, much like the old days of eInk displays (oh, wait, they still have those, don't they!). What this means in practical terms is that, although you can now include very large images in your Kindle ebooks, you will want to carefully manage the ultimate file size at which each image can be delivered, since you can no longer assume Amazon will do this for you. If you include 5 Mb image files, they will now send those full size images to the end user's HD device, for good or ill.

Which brings me to my final point, and that concerns Amazon's delivery fees. One major concern with image file size increasing has been the prohibitive cost of a .15 cent per megabyte bandwidth charge incurred under the 70% royalty option, since a single 5 Mb image would therefore cost .75 cents to send! But surprisingly, this does not prove to be the case.

While the KDP preview file I downloaded today weighed in a 45.5 Mb, the "book size after conversion" listed on the product pricing page was only 3.36 Mb - the size Previewer's log listed as the "deliverable file size" after conversion. This makes the total download fee just .50 cents for the full size HD ebook, leaving $1.74 profit for a $2.99 list price title. Amazon has essentially decided to subsidize content for the HD devices by only charging for the lower file size. This makes perfect sense, since you cannot rightly charge the higher fee for files sent to lower resolution devices, and charging different rates for different devices is simply impractical (not to mention a bookkeeping nightmare). It's also probably why they haven't bothered to publicize this much. Or at all.