How do you figure?

Scott O’Hara digs into the <figure> and <figcaption> elements. Gotta love a good ol’ HTML deep dive.

I use these on just about every blog post here on CSS-Tricks, and as I’ve suspected, I’ve basically been doing it wrong forever. My original thinking was that a figcaption was just as good as the alt attribute. I generally use it to describe the image.

<figure> <img src="starry-night.jpg" alt=""> <figcaption>The Starry Night, a famous painting by Vincent van Gogh</figcaption>

I intentionally left off the alt text, because the figcaption is saying what I would want to say in the alt text and I thought duplicating it would be annoying (to a screen reader user) and unnecessary. Scott says that’s bad as the empty alt text makes the image entirely undiscoverable by some screen readers and the figure is describing nothing as a result.

The correct answer, I think, is to do more work:

<figure> <img src="starry-night.jpg" alt="An abstract painting with a weird squiggly tree thing in front of a swirling starry nighttime sky."> <figcaption>The Starry Night, a famous painting by Vincent van Gogh</figcaption>

It’s a good goal, and I should do better about this. It’s just laziness that gets in the way, and laziness that makes me wish there was a pattern that allowed me to write a description once that worked for both. Maybe something like Nino Ross Rodriguez just shared today where artificial intelligence can take some of the lift. But that’s kinda not the point here. The point is that you can’t write it once because <figcaption> and alt do different things.

Direct Link to ArticlePermalink

The post How do you figure? appeared first on CSS-Tricks.

Using Artificial Intelligence to Generate Alt Text on Images

Web developers and content editors alike often forget or ignore one of the most important parts of making a website accessible and SEO performant: image alt​ text. You know, that seemingly small image attribute that describes an image:

​​​<img src="/cute/sloth/image.jpg" alt="A brown baby sloth staring straight into the camera with a tongue sticking out." >

A brown baby sloth staring straight into the camera with a tongue sticking out.
📷 Credit: Huffington Post

If you regularly publish content on the web, then you know it can be tedious trying to come up with descriptive text. Sure, 5-10 images is doable. But what if we are talking about hundreds or thousands of images? Do you have the resources for that?

Let’s look at some possibilities for automatically generating alt text for images with the use of computer vision and image recognition services from the likes Google, IBM, and Microsoft. They have the resources!

Reminder: What is alt text good for?

Often overlooked during web development and content entry, the alt​ attribute is a small bit of HTML code that describes an image that appears on a page. It’s so inconspicuous that it may not appear to have any impact on the average user, but it has very important uses indeed:

  • ​​Web Accessibility for Screen Readers: Imagine a page with lots of images and not a single one contains alt​ text. A user surfing in using a screen reader would only hear the word “image” blurted out and that’s not very helpful. Great, there’s an image, but what is it? Including alt​ enables screen readers to help the visually impaired “see” what’s there and have a better understanding of the content of the page. They say a picture is worth a thousand words — that’s a thousand words of context a user could be missing.
  • Display text if an image does not load: The World Wide Web seems infallible and, like New York City, that it never sleeps, but flaky and faulty connections are a real thing and, if that happens, well, images tend not to load properly and “break.” Alt text is a safeguard in that it displays on the page in place of where the “broken” image is, providing users with content as a fallback.
  • ​​SEO performance: Alt text on images contributes to SEO performance as well. Though it doesn’t exactly help a site or page skyrocket to the top of the search results, it is one factor to keep in mind for SEO performance.

Knowing how important these things are, hopefully you’ll be able to include proper alt​ text during development and content entry. But are your archives in good shape? Trying to come up with a detailed description for a large backlog of images can be a daunting task, especially if you’re working on tight deadlines or have to squeeze it in between other projects.

What if there was a way to apply alt​ text as an image is uploaded? And! What if there was a way to check the page for missing alt​ tags and automagically fill them in for us?

There are available solutions!

Computer vision (or image recognition) has actually been offered for quite some time now. Companies like Google, IBM and Microsoft have their own APIs publicly available so that developers can tap into those capabilities and use them to identify images as well as the content in them.

There are developers who have already utilized these services and created their own plugins to generate alt​ text. Take Sarah Drasner’s generator, for example, which demonstrates how Azure’s Computer Vision API can be used to create alt​ text for any image via upload or URL. Pretty awesome!

​​See the Pen
​​Dynamically Generated Alt Text with Azure's Computer Vision API
by Sarah Drasner (@sdras)
​​on CodePen.

There’s also Automatic Alternative Text by Jacob Peattie, which is a WordPress plugin that uses the same Computer Vision API. It’s basically an addition to the workflow that allows the user to upload an image and generated alt​ text automatically.

​​Tools like these generally help speed-up the process of content management, editing and maintenance. Even the effort of thinking of a descriptive text has been minimized and passed to the machine!

Getting Your Hands Dirty With AI

I have managed to have played around with a few AI services and am confident in saying that Microsoft Azure’s Computer Vision produces the best results. The services offered by Google and IBM certainly have their perks and can still identify images and proper results, but Microsoft’s is so good and so accurate that it’s not worth settling for something else, at least in my opinion.

Creating your own image recognition plugin is pretty straightforward. First, head down to Microsoft Azure Computer Vision. You’ll need to login or create an account in order to grab an API key for the plugin.

Once you’re on the dashboard, search and select Computer Vision and fill in the necessary details.

Starting out

Wait for the platform to finish spinning up an instance of your computer vision. The API keys for development will be available once it’s done.

​​Keys: Also known as the Subscription Key in the official documentation

Let the interesting and tricky parts begin! I will be using vanilla JavaScript for the sake of demonstration. For other languages, you can check out the documentation. Below is a straight-up copy and paste of the code and you can use to replace the placeholders.

​​var request = new XMLHttpRequest();'POST', 'https://[LOCATION]/vision/v1.0/describe?maxCandidates=1&language=en', true);
request.setRequestHeader('Content-Type', 'application/json');
request.setRequestHeader('Ocp-Apim-Subscription-Key', '[SUBSCRIPTION_KEY]');
request.send(JSON.stringify({ "url": "[IMAGE_URL]" }));
request.onload = function () { var resp = request.responseText; if (request.status >= 200 && request.status < 400) { // Success! console.log('Success!'); } else { // We reached our target server, but it returned an error console.error('Error!'); } console.log(JSON.parse(resp));
}; request.onerror = function (e) { console.log(e);

Alright, let’s run through some key terminology of the AI service.

  • Location: This is the subscription location of the service that was selected prior to getting the subscription keys. If you can’t remember the location for some reason, you can go to the Overview screen and find it under Endpoint.
  • ​​

Overview > Endpoint : To get the location value
  • ​​Subscription Key: This is the key that unlocks the service for our plugin use and can be obtained under Keys. There’s two of them, but it doesn’t really matter which one is used.
  • ​​Image URL: This is the path for the image that’s getting the alt​ text. Take note that the images that are sent to the API must meet specific requirements:
    • File type must be JPEG, PNG, GIF, BMP
    • ​File size must be less than 4MB
    • ​​Dimensions should be greater than 50px by 50px

Easy peasy

​​Thanks to big companies opening their services and API to developers, it’s now relatively easy for anyone to utilize computer vision. As a simple demonstration, I uploaded the image below to Microsoft Azure’s Computer Vision API.

Possible alt​ text: a hand holding a cellphone

​​The service returned the following details:

​​{ "description": { "tags": [ "person", "holding", "cellphone", "phone", "hand", "screen", "looking", "camera", "small", "held", "someone", "man", "using", "orange", "display", "blue" ], "captions": [ { "text": "a hand holding a cellphone", "confidence": 0.9583763512737793 } ] }, "requestId": "31084ce4-94fe-4776-bb31-448d9b83c730", "metadata": { "width": 920, "height": 613, "format": "Jpeg" }

​​From there, you could pick out the alt​ text that could be potentially used for an image. How you build upon this capability is your business:

  • ​​You could create a CMS plugin and add it to the content workflow, where the alt​ text is generated when an image is uploaded and saved in the CMS.
  • ​​You could write a JavaScript plugin that adds alt​ text on-the-fly, after an image has been loaded with notably missing alt​ text.
  • ​​You could author a browser extension that adds alt​ text to images on any website when it finds images with it missing.
  • ​​You could write code that scours your existing database or repo of content for any missing alt​ text and updates them or opens pull requests for suggested changes.

​​Take note that these services are not 100% accurate. They do sometimes return a low confidence rating and a description that is not at all aligned with the subject matter. But, these platforms are constantly learning and improving. After all, Rome wasn’t built in a day.

The post Using Artificial Intelligence to Generate Alt Text on Images appeared first on CSS-Tricks.

Some Things About `alt` Text

I’m sure you know about alt text. It’s the attribute on the image tag that has the important task of describing what that image is for someone who can’t see it for any reason. Please use them.

I don’t want to dimish the please use them message, but some interesting alt-text-related things have come up in my day-to-day lately that are related.

When you don’t

Hidde de Vries wrote You don’t always need alternative text recently:

But when an icon has a word next to it, for example ‘Log out’, the icon itself is decorative and does not need an alternative text:

<button type="button"><img src="close.svg" alt="" /> Close</button>

In this case we can leave the alt attribute empty, as otherwise a screenreader would announce ‘button – close close’.

I would think in a perfect world because that icon is entirely decorative that applying it via CSS would be ideal, but the point stands: if you have to use an image, alt text hurts more than it helps here.

Can we get it for free?

Computers are pretttttty smart these days. Perhaps they could look at our images and offer up descriptions without us manually having to type them.

Sarah’s demo uses a Computer Vision API to do that.

What about figcaption?

I hate to say it, but I’m not particularly good at writing alt text descriptions for images in blog posts right here on CSS-Tricks. It’s a problem we need to fix with process changes. We do often use <figcaption> though to add text that’s related to an image. The way that text is often crafted feels like alt text to me. It describes what’s going on in the image.

I was asking around about this, and Zell Liew told me he does the same thing:

I actually have the same question. Most of my figcaptions are used to describe the image so readers understand what the image is about.

In my mind, I figured it would be worse to drop the exact copy from the figcaption into the alt text, as someone who was reading alt text would then essentially read the same description twice.

I also talked to Eric Bailey who had an interesting idea.

<figure> <img src="screenshot.png" alt="Screenshot of Chrome displaying a split view. On the left is a page full of image thumbnails comparing pre and post-optimization filesize. On the right is Chrome developer tools showing paint rasterize duration for the images. With a 6x CPU slowdown, the longest Paint Raster took 0.27ms, AKA 0.00027 seconds."> <figcaption aria-hidden="true"> With a 6x CPU slowdown, the longest Paint Raster took 0.27ms, AKA 0.00027 seconds. </figcaption>


  • Preserves figure styling
  • Avoids nulling alt, which can be problematic for some screen readers
  • Keeps the description close to the content and communicates point to SR users
  • Communicates significant takeaway to visual readers without duplicating reading for SR users
  • Uses an aria property designed to be used outside of forms

I’d stress that he considered this just an idea and it hasn’t been heavily vetted by the larger accessibility community. If there are any of you out there reading, what do you think?

Eric’s demo used a more verbose alt text than the figcaption, but it seems like the pattern would be fine even if they were identical.

Little reminder: Twitter has alt text

But you need to enable the feature.

The post Some Things About `alt` Text appeared first on CSS-Tricks.