Thursday, January 22, 2015

Kindle Textbook Creator Breakdown

An astonishing new app was released by Amazon today called the Kindle Textbook Creator that creates yet another entirely new ebook format for the Kindle with these astounding "enhanced" features:
  • You can highlight text!
  • You can add notes!
  • You can add charts!
  • You can use the dictionary!
  • You can search Wikipedia!
  • You can sync across devices!
But wait, doesn't the old Kindle format do all that? Oh, but there's more!

You can also add "flashcards" (well, not yet, but someday...), and columns (just like PDF), and math equations (from the font embedded in the PDF), and graphs (copied as an image from the PDF) and all you have to do is load in a PDF! So, basically, the "new" .kpf (Kindle Package Format) is a .pdf file with the file extension changed. You put in a PDF, and out comes a PDF clone.

So now let's look at what the new format cannot do, or is restricted to:
  • The input can only be a PDF
  • The output file is not reflowable
  • There are no text or image overlays
  • There is no MathML support (since ePub3 is not a valid import format)
  • Output file cannot be read or opened directly in a Fire device or Kindle app
  • Output file can only be uploaded to KDP for publishing
  • The file created can only be sold on Amazon, per terms of service
At present there is no way to actually add the "flashcards" that I can find. There are, in fact, only four options available at all. Two of these exist on the Edit menu, and consist of "Insert Page(s)" and "Delete Pages". The other two options are available from the File menu or from little icons in the upper right corner:
The Preview option brings up the built-in Inspector, which is a variation of Previewer, with basic set of menu options:

On the right the Device drop-down menu shows that there are currently four available preview modes: Fire HDX & HDX 8.9, iPad, and Android Tablet. Curiously, the Kindle Voyage and DX are also listed (though greyed out), even though the release notes specifically state that these "e-Textbooks" will not be available on the Kindle eInk devices.

The Package option outputs your file to the .kpf format for upload to KDP. You can also save your project as a .kcb file for later editing (make sure you do this as you cannot re-open the packaged .kpf file in KTC, and will have to start over if you did not save a project file!). Both of these files can be opened and viewed using a zip extractor and/or a text editor, but most of the content is encrypted gibberish, and what can be read is essentially useless - the "book.kcb" file contains a handful of lines with some obscure metadata and path reference elements.

Also, the file that is output from KTC is virtually the same size as the input PDF (5.64 Mb > 5.62 Mb in my test), showing again that essentially it is still more or less the same file, except that now your "PDF" can only be read on a Kindle and nowhere else (and only on some Kindles, at that).

The Inspector does not actually allow you to "preview" any of the features touted in the press release as benefits of this new format (i.e highlighting, dictionaries, etc.). In fact, the only "interactivity" in the Inspector is to change the page zoom in increments from 100% to 400% - which is essentially irrelevant. The live text layers I created in the PDF were not active in the Inspector, though I must presume they would be in the published file.

Unfortunately there is no way to know this, since after actually uploading the file to KDP you can only preview it in the online previewer - there is no download button or link to save the file for manual preview on a device or app as there is with all other Kindle formats. Therefore, I cannot speak to the quality or functionality of the final published content, as I have no intention of ever using this to produce an actual book that I would want someone to read. You are free to do so if you like, but I can see no good reason to bother with it at this point. Bear in mind that KTC is still technically in beta, although since this is a public release that more or less overrides its beta status.

There is a User's Guide available to download from the KTC page, but it tells you very little (since there is, in fact, very little to be told). The FAQ, however, is fairly lengthy this time, and includes a few important bits of information, such as:
Q10: Can I sell books I create using Kindle Textbook Creator outside of the Kindle store?
To which the answer is "no" - followed by a link to the license agreement that says so in perfect legalese.

On the positive side, Amazon learned from the Kids Book Creator debacle and added an Undo button, so that's something I suppose (there is also a "redo" button in case you change your mind again, but I recommend the "uninstall" option instead).

Tuesday, December 23, 2014

Kindle Publishing Guidelines 2014.3

The latest update to the Kindle Publishing Guidelinesis out, with a lengthy change log that includes a few important changes. While consisting primarily of editorial revisions to specify which Kindle device a given formatting recommendation applies to, there are two specific changes to Kindle content production that are significant, these being sections 4.3.7 and 4.3.9 as given in the Revision Notes shown above.

4.3.7 Recommendation #7: Do Not Include an HTML Front Cover

The first of these reverses a long-standing error in Amazon's Kindle production policy. Until now this section heading read Include an HTML Front Cover, while it now emphatically says not to (as I have long advocated). Including one has always resulted in two cover images appearing, causing reader confusion due to the apparent unresponsiveness of the first page turn. Moreover, there has never been a good reason to include an HTML cover page, since all Kindle devices and apps render the jpeg cover image correctly, as well as using it for the bookshelf image.

As the Kindle Publishing Guidelines itself now states:
While Amazon previously recommended an HTML front cover page for fixed-format books, this is no longer necessary. 
Kindle books should only have one visible JPEG cover. This cover should be a high-resolution JPEG image that has the same level of quality as the subsequent pages. Any instances of HTML cover pages should be deleted to avoid a repetition of the cover image.
Presumably the HTML cover page was originally included in order for the Guide to point to the cover as the first page the reader sees, if this was so desired (since it could not point directly to an image at that point, as it now can). But since the implementation has been inconsistent, opening to the jpeg image in many cases, as well as the reader being able to swipe back to the cover image anyway, this has caused confusion and frustration for many readers.

The recommendation to include a high resolution cover image is now all the more important, since it will be rendered as a full page image on first opening.

4.3.9 Recommendation #9: Do Not Include Start Reading Location

This is an entirely new addition which addresses an ongoing issue with the first page to which a Kindle ebook opens on first reading. This has been erratic and seemingly random, since the same ebook would open to different pages on different Kindle iterations, depending on a number of confusing factors (including, among other things, the appearance or absence of a "toc" entry in the Guide, as detailed on pages 74-5 of my Kindle formatting manual).

While Amazon has repaired several of these errant instances, some have continued to persist (such as the perplexing inability of the "Go to Beginning" entry to open at Page 1). This issue is now eliminated with the newest Guideline recommendation:
In Kindle fixed-format books, the OPF file should not include the start reading location (”Go to Beginning”) guide item. Amazon now sets this guide item to the JPEG cover for Kindle fixed-format books.
All fixed layout Kindle ebooks will now open to the cover image (which should now only be a jpeg image), rather than to the title page, the table of contents, the first page after the table of contents, or any other random location in the publication. This provides for a consistent user experience across devices and platforms, as should have been the case all along.

Note that Section 3.5.1 on "Recommended Guide Items" still lists the "Go To Beginning" entry as a valid item. This is likely an editorial oversight, but since the removal of the start reading Guide item is only a recommendation, the entry is technically still valid, though not advised.

Note also that you can still include a "bodymatter" element for the start reading location in the Landmarks section of a nav doc, though this will not affect the page to which the book first opens, but only create a linked menu entry to the chosen page.

2.2.2.3 Using KindleGen

Among the other changes made in this edition is the inclusion of Dutch as an optional language in KindleGen (using the locale option nl during conversion).

3.6.3 Image Guideline #3: Use Color Images

Removed the line describing the difference between eInk and color tablet devices, and added a statement that photographs must be in the jpeg format. Specifically, the line removed stated that:

The Kindle e Ink devices currently have a black and white screen, but color is available on the Kindle Fire, Kindle for iPhone, and Kindle for PC.
This is curious, since the removal of the reference to a greyscale display hints at the coming of a color eInk screen. Although this has long been awaited, to date there is no evidence that a reflective color display is forthcoming, and we have already missed this year's pre-holiday release window, so presumably it won't appear for at least another year.

The appended statement that photos must be formatted as jpegs is followed by further image clarifications in the next section.

3.6.5 Image Guideline #5: Use GIF or PNG for Line-Art and Text

This entry has been extensively revised to make it adamantly clear that line-art and text images should not be formatted as jpegs, which blur sharp edges when adding compression, but should instead be embedded as either gifs or png files, which preserve crisp edges - and even enhance them, in the case of gifs, by reducing color values and grayscale contrast (as the added line "including black-and-white drawings" makes clear).

One line has been removed here that bears comment:
The automatic conversions applied by KindleGen are best avoided.
This statement was always inaccurate in this context, since all supported image formats are converted to jpegs during the KindleGen conversion, and therefore cannot be avoided. What was meant instead was that the cleanest possible image should be input in order for KindleGen to apply the best conversion possible. If an already compressed jpeg is embedded, its quality will only get further reduced by the additional compression applied by KindleGen when converting to the low Mobi 7 image standards.

This requirement is emphatically repeatedly in this section several times, culminating in the conclusion that
Amazon insists on GIF or PNG file formats for line-art.
Finally, a lengthy section has been added to the description of the MINIMUM size requirements for lines of text contained in images (which is 6 pixels for a lowercase "a"), including the addition of a new example image:
The added text states that the image should be taller than the 6 pixel text itself, such that
the image should be at least 45 pixels in height so that it displays proportional to surrounding text content.
This assumes, of course, that the surrounding text content has not been modified by the user's font size and line height settings, but the intention is clear: make your text images big enough that they can be clearly seen, and leave some margin space around them so they preserve the line height and don't encroach upon surrounding text.

6 Audio and Video Guidelines

Several additions have been included to reiterate the fact that, no, audio and video is not supported on eInk devices, and KDP does not accept Kindle Editions with audio/video content included. Still.

This is simply a shortcoming of the slow refresh rates of eInk screens (and one of the primary reasons reflective color displays have not been adopted), although why audio is forbidden is beyond me. Probably for the same reason Amazon stopped bothering to include speakers, or even a headphone jack, on the eInk devices. Or perhaps because of it.

9 Kindle Best Practices

Here again some lines have been removed, and the terms "e Ink" and "tablet" inserted to more clearly differentiate between the two. There is also a revision of the formats supported for viewing on devices and in Previewer, via the removal of two lines.
The Kindle Fire device view displays the content in Kindle Format 8.
This applies to Previewer, and was probably removed due to the fact that it is no longer relevant, since virtually all Kindle devices now support KF8, and Previewer displays content as such.
You can test Mobi 7 content on a Kindle e Ink device and on Kindle applications for PC/Mac/Android.
This line was made more or less redundant by the fact that Mobi 7 content is now only sent to the oldest of the old Kindle devices, and is very likely soon to be eliminated. Moreover, there is no way to actually chose which version of the converted file is being tested, since the device reads whichever one it has support for. Again, virtually all Kindle devices now read KF8, so the distinction is no longer relevant.

The subsequent line that stated KF8 could only be tested on the Kindle Fire has also been appended to include the eInk devices, but interestingly not the apps. This implies that you can no longer use (or rely on using) the desktop and mobile apps for testing files, which is sound advice, since they are the least consistent in rendering content correctly, or supporting features. Best Practice is, and has always been, to test on an actual Kindle device, followed by Previewer, and only as a last resort to use a desktop or mobile app (although the Android app is better than the other apps by far, as far as feature support goes).

All in all, these updates pave the way for future moves away from the outdated Mobi 7 format and into feature-rich KF8 support across the board (more or less), and address some outstanding issues with the format itself in order to make the reading experience more consistent.

Saturday, December 13, 2014

iBooks Image Size Increased

Although they have not yet updated their iBooks Asset Guide to reflect the change, Apple has just announced an increase to the total pixel limit allowed for interior images in ebook files published through iTunes Connect, upgrading it from 3.2 to 4 million pixels.

This follows the increase in August of last year from 2 to 3.2 million pixels, and at long last brings the overall image size allowed more in line with the actual size of the device on which they are intended to be viewed.

The increase to 3.2 million from 2 million pixels was itself, of course, a long overdue attempt to address a contradiction in Apple's recommended image size as given in the iBooks Asset Guide. With a resolution of 2048 x 1536 since the 3rd generation iteration of the full-size iPad (giving it a total pixel count of 3,145,728), the long-standing limit of 2 million pixels was nowhere near enough to fill the screen with a full bleed image in portrait orientation, let alone to zoom in for greater detail, as the Asset Guide somewhat ironically recommends:
Apple recommends providing images that are at least 1.5 times the intended viewing size up to a maximum of 3.2 million pixels.
A quick bit of math will show that 1.5 times the standard iPad display resolution is 4,718,592 pixels. So while the new 4 million pixel limit does not provide for quite that much leeway, it gets us very nearly there, and at least allows for full bleed images that can be zoomed to some degree without completely losing fidelity. The official announcement for the newest update reads as follows:
Increased Pixel Limit for Book Images
The pixel limit for all images within a book has been increased to 4 million pixels. This limit does not apply to images delivered separately from the book, such as cover art or screenshots.
Amazon, of course, has also recently increased the allowed image size in Kindle ebooks, although they use an overall image file size in megabytes rather than a pixel dimension limit. This allows for a much higher image resolution to fill the bigger 2560 x 1600 HDX 8.9 device, which boasts over 4 million pixels, before zooming (4,096,000 to be precise). By comparison, that would require an image containing 6,144,000 pixels in order to zoom to 1.5 times without interpolation! This is why the individual image limit in Kindle files has been increased to 5 MB. Converted mobi files for upload to KDP, however, are limited to an overall file size of 650 MB, which puts something of a practical limitation on the included image files (130 5Mb files to be precise, which is admittedly more than ample for most projects).

Conversely, by employing a pixel limit, Apple are restricting the total image dimensions, but not the image quality itself, which can be embedded with no compression whatsoever, if so desired, up to a total ebook file size limit of 2 GB (which is a limit imposed by the zip format itself). So while the new 4 million pixel limit does not quite allow for full 1.5 times zooming with pixel-perfect accuracy, it does come very close, and the absence of a file size limit for individual images allows for the highest quality setting possible, which might be of great benefit for art books and photography collections (in case you want to see those brush strokes on the Mona Lisa, for example).

By the way, for those wanting to know, 2309 x 1732 would be the largest dimension image you can now insert into an iBooks file at the standard 4:3 aspect ratio of the iPad display. That gives you an image containing 3,999,188 pixels. Just one pixel bigger at 2310 x 1733 puts you over at 4,003,230 pixels. A nice round numbered 2300 x 1725 gives you an image with 3,967,500 pixels.

Thursday, November 13, 2014

BISG Field Guide to Fixed Layout Update Errors

The Book Industry Study Group (BISG) this week released a revised edition of theirField Guide to Fixed Layout for Ebooks (Version 1.2), but it is still fraught with errors not fixed from the last edition, as well as adding new confusion via poor editing and worse research. You can download it for free from the link above (once you register for an account), which is worth doing, as it does contain some useful information in among the rest; you just have to know which is which.

A new section has been added, for example, called "Authoring Tools & Limitations" which, sinceTools is plural, would presumably purport to outline a number of the primary applications currently in use for the production of fixed layout ebooks, such as iBooks Author, Jutoh, CircularFlo, MadeFire, or at the very least the two free programs offered by Amazon, Kindle Comic Creator and the Kindle Kids Book Creator. But in fact, the only tool actually discussed is InDesign CC's ePub export feature. No other program is discussed, or even mentioned. That said, the extensive list of InDesign's limitations is well worth a read, since it details exactly the reasons why I do not use InDesign for ebook production, nor recommend it, at least for the creation of Kindle fixed layout source files.

And on the topic of KF8, the following are just the primary errors to be found in the section on the Kindle fixed layout format...

1.) In the information regarding the Orientation Lock options (page 26, item #3), it states that none is a valid option, along with portrait and landscape, but then later contradicts this with a paragraph stating (incorrectly) that "you cannot have a book that can be viewed in both portrait and landscape modes," which of course, is exactly what the none value is for. Moreover, the statement that "Orientation is a required value" is only partly correct, since it is optional in comics, and only required for children's ebooks, but in either case can be set to none (which is, in fact, the default value). Granted, auto-orientation does not function on many Kindle devices, but it does on some, and it is the none value that allows for this.

2.) Item #4 states incorrectly that the values of the "Book Type" entity "can only be set to children or comic." Since this entity is optional, it can also be left out (or set to none), in either case of which the effects of the fixed layout functions vary greatly from those of either of the other two book-types. See my Fixed Layout Functionality Chart for specific details on this, if you have not already. There is good reason not to include either book-type.

3.) The last sentence on the bottom of page 26 cuts off mid-steam at the top of the following page, but seems to suggest that two-page spreads are possible only with the comic book-type. This is incorrect, since it works on some devices and not on others for both children's and comics, as well as those containing no book-type value (or a value of none), as can be seen in item #2 in the Fixed Layout Functionality Chart referenced above. However, the comic book-type does support two-page spreads more widely than either of the other two options, though not consistently. While I am sure it is beyond the scope of the Field Guide to provide these specifics, such universal statements as those given tend to be misleading, and are therefore less than helpful.

4.) The Region Magnification entity is irrelevant, since the value is set automatically by Kindlegen based on the presence or absence of region mag code in the converted file itself. The value will be set to the correct value regardless of what the content creator enters in the OPF - even, in fact, if no value or entity is added to the metadata at all. Therefore, there is little point in adding it manually, one way or the other.

5.) The statement that "KF8 fixed-layout format lacks pan and zoom" is incorrect. No, you cannot pinch and zoom pop-up regions, but you can pinch and zoom background images inserted using the <img src= > embedding method after double tapping the image (while those referenced via CSS div id's are locked). This is shown by Amazon's own Comics Sample file, which uses the unlocked insertion method almost exclusively. Moreover, pinch and zoom is a fundamental feature of the Virtual Panels function for KF8 comics that don't include any custom magnification regions: the pinch and zoom is automatically applied by Kindlegen without any active effort on the content creator's part. Refer to Item #4, Notes 6 & 7, on the Fixed Layout Functionality Chart for specific details on this.

This is just a sample of the errors and misleading information found in BISG's Field Guide. For this publication to be truly useful each of the sections and sub-sections provided needs to be greatly expanded, with detailed discussions of each specific topic, and the particular details pertaining to each platform and format, particularly if they actually expect readers to donate up to $50, as prompted by the "Pay What You Want" radio buttons on the download page. I'm not sure who in the publishing world would feel satisfied paying that much for so little rudimentary information when much more detailed data is available on blogs such as this for free. But it's your money, you can spend it as you like. I recommend the free option.