The Magic of Generative Fill

 

"Distraction" Removal Strasbourg France

My intention in this article is to discuss the marvelous new Photoshop tool, Generative Fill, but first I feel I need to explain and apologize for my prolonged absence (until a week ago) from publications in this blog.

I am a little surprised and a lot ashamed that it has been just over two years since my last post to my digital photography blog. I understand some, but not all, of the reasons for my absence.  I have always said that I do the blog primarily for my own fulfillment, a way to force myself to consider the challenges of digital photography in a more formal way.  I hope that my discussions have been helpful to others, although I must observe that my ghosting has not led to an uproar of disappointment from my “fans”.

 

The reasons for my absence are many.  Certainly, I must blame, as everyone does, the pandemic, although you might expect that all that free time should have led to more output.  Over the years I always did my best writing while settled at a table in my favorite coffee house and it just hasn’t been the same staring at the monitor isolated in my studio.  The good news is that I started this blog back at a table in the expanded Brewbakers. I did expect that my retirement would have led to more free time to devote to writing but there have been various competing priorities. 

Courthouse Butte Sedona Arizona

Several trips, including to Iceland, Ireland, Arizona and up the Rhine River have added volumes of images requiring editing, and of course the documentation of our two grandchildren have contributed great piles of pixels.  All excuses aside, I must acknowledge that a major reason for my silence must be ascribed to my own profound laziness.  There is something about having all this free time that encourages sloth. My pace is generally slower.  With all that free time tomorrow, it is always easy to put off what I should have done today. 

 

Enough excuses.  In the last two years the light has not changed and the beauty of New England remains remarkable.  What has changed are the tools that we have available to capture that beauty.  Successive updates to Adobe Lightroom have made it a more capable image editing tool while retaining its file management and organizational strengths.  It is possible to rely solely on Lightroom for your image management and processing, but Photoshop still is the gold standard and offers image editing capabilities that are still not present in Lightroom.  AI supported Generative Fill (GF) is one these powerful capabilities. I should add that GF is only available in the newer versions of Photoshop.  You won’t find it in Lightroom (yet).

 

Marshall Point Light

Added Gulls & Lobster Boat

 Generative Fill (GF) is not a perfect tool, but in many situations, it is almost magical.  Much of the attention given to Generative Fill is based on its ability to insert content into images.  By selecting a portion of an image GF can add almost anything that you may ask.  In my sunset picture of Marshal Point Light, I selected an area in the right upper sky and asked for three seagulls at sunset.  Each time you press the “Generate Button”, after a short period of “thinking”, the tool provides three choices. In this case one of the initial choices worked reasonably well.  Of course, I couldn’t stop there and added a lobster boat on the left.  GF gets these images from Adobe’s extensive image library, and it often    works quite well.  Of course, there is no roadblock to keep you from going too far.

Adde Elephant

 

In this case, an elephant parading among the rocks.  The power of GF to insert content is remarkable and, although it often has an artificial quality, it will certainly get better with further refinement. It is a lot of fun, but for me, I am uncomfortable manipulating the content of my images to this degree and I essentially never use this feature of GF

Generative fill has a couple of other tricks that I find quite useful.  First it can be used to expand the size of image backgrounds.  In this case my Bluebird was cropped too tight to his beak.  After expanding the canvas size to the right, I selected that area, pulled up Generative Fill and triggered it without putting anything in the text box.  The tool nicely sampled the surrounding area and generated a smooth consistent pattern for the background.  GF does a remarkable job expanding the edges of images whether it is sky, grass, water or whatever else you need.

Distraction Removal

Peggy's Cove Light Nova Scotia
Why Do They Watch Beside the Light

By far my most frequent use of Generative Fill is in the removal of distracting content from my images. I have always been compulsive about removing distraction to make my images simpler while directing attention to the key elements of the composition.  I have spent hours with my cloning tool trying to seamlessly eradicate stray rocks and branches peaking in from the edges of my images and most of all, trying to erase all the people who loiter about blocking the natural beauty.

People Removed the
Old Fashion Way

It’s not that I necessarily dislike people. They just don’t belong in my images. I contend that when we look at a beautiful view our minds tend to eliminate the crowds that inhabit the space. In a photograph those interlopers stand out and can’t be ignored.  My view is that removing these distractions brings the image back to the way we originally perceived it.  With that justification, I want to show how the use of Generative Fill has vastly increased the ease of removing distractions of all sorts.  I can illustrate this best by looking at a few images from my recent cruise up the Rhine River from Amsterdam to Switzerland.  On the trip I was frequently gifted with beautiful scenes of classic villages and ancient cathedrals.  Spectacular but also overrun with wandering groups of tourists.  It is easy to be frustrated by the gaggles, but I always had to remind myself that I was one of the tourists as well.

In the past I would use cloning to carefully remove a few of the most intrusive humans and other distractions, always balancing the severity of the distraction against the time required to meticulously scrub away the contamination. Often the decision was based on the complexity of the background that must be duplicated to fill in the space opened by the removal. In images that I found especially powerful, like the Peggy's Cove Light above, the time required for removal of distractions was more justified and I could spend hours cleaning things up to let the image shine through.

 

Crowded Old Town Strasbourg

Generative Fill has made this process much easier, reducing to a fraction the time required to clean up even the most cluttered image. 

 

Partial Crowd Selection

To use GF, you simply make a selection around the offending area (as above) and then open the Generative Fill tool in the “Edit” drop down menu in Photoshop. Click “Generate” without entering anything in the dialog box and wait for the results.  As always, GF will offer three choices based on the surrounding pixels.  Some can be bizarrely awful, but others are surprisingly good matches.  If you don’t like any of the choices, you can hit the “Generate” button again and get three more possibilities. 

Generative Fill of Selection

I’m not always happy with the first results, but more often my response is “WOW, how did it do that”.  Using GF, the process of cleaning an image is so fast and accurate that I find myself eliminating distractions that I would have previously let slide because they were either too minor or too complicated to work on.  

Further Removal of Distractions

Cologne Cathedral 

Sometimes GF seems to have problems getting things right. GF can dip into the Adobe Image Database to pull up crazy stuff to fill the gaps. You don’t want a poster board filled with gibberish replacing the person in front of the cathedral altar. In these situations, redrawing the selection may help.  

Crowded Bridge Strasbourg

For large or complicated areas breaking the target into a few smaller selections can help GF get a better result.   Here I had to do separate fills on the bridge and the reflection.  The important point is not that the GF results can go awry, it is how often it fills your selection with remarkably appropriate content.

Bridge is Clear

 

Spalen Gate Basel Switzerland

As I worked my way through my thousands of Rhine River images, I was amazed how many times I used Generative Fill to clean up my pictures. Amsterdam and the Rhineland are full of majestic landscapes, quaint villages and breathtaking cathedrals and Generative Fill allowed all these sites to shine past the distractions without adding frustration and hours to my editing time. Give it a try.  Once you get a feel for it, you’ll find GF is amazingly powerful and a lot of fun.

 

Also, check out my Gallery of images from Amsterdam and the Rhine 

 Jeff Newcomer
www.partridgebrookreflections.com

Iceland and Introduction to Adobe Lightroom Course

Gullfoss Falls

I’ve got a couple of major projects competing for my attention right now.  Actually three, but my taxes are nearly complete.  It is always surprising when I discover that I made money.  As usual, last year it was not much, not enough to live on, but for a landscape photographer anything in the positive is good news.  The other two demands on my attention are even better.

Drowning Pool, Pingvellir National Park, Iceland

First Susan and I just returned from a to brief, long-weekend, trip to Iceland.  I have always wanted to shoot in Iceland, and we managed to stuff a lot in a short period of time.  This tour convinced me that I must go back for a longer visit, likely in a warmer season.  I came home with over 400 images to work on, and that has reminded me about how dependent I am Adobe Lightroom as an essential part of my picture editing workflow.

Shameless Self-Promotion

Pingvellir Gorge

I am excited about my up-coming Introduction to Adobe Lightroom Classic Course.  The course will meet on six consecutive Tuesday evenings from March 3^rd through April 7^th at 6pm.  It will be a small group gathered around my dining room table in Spofford New Hampshire.  There are a couple of spaces still open, so get in touch if you are interested in learning about improving and organizing your images with this amazing program.  For more information, check out my article about my spring teaching schedule or give me a call at 603-363-8338.

Given the Power of Lightroom, Is Photoshop Necessary?

People often ask how much of my image editing is done in Lightroom and how much in Photoshop.  I feel strongly that, given the power and ease of use, for the majority of digital photography enthusiasts, Lightroom is all they will need to get started with image editing.  In my own workflow, I use Lightroom for my RAW editing, but I still routinely finish by moving my images to Photoshop for varying degrees of final touch-up.  Often after my full Lightroom adjustments, there is little to do.  I scan the image for small imperfections, dust spots or other distractions that I find easier to remove using Photoshop’s healing bush or cloning tool.  Subtle local adjustments in lighting or color can be applied more precisely using layer masks, and of course, manipulations of pixels can best be performed from within Photoshop. 

Pingvallavatn Lake

I almost always find something to do in Photoshop to complete my image, but how much of a difference does it really make and how much time do I spend using each of these often-complimentary programs?  I usually tell people that I get 80-90% of my image editing done in Lightroom, but that is just a guess.  I decided to look at the question in more detail as I edited some of my Iceland pictures.

Steaming, Geysir Hot Springs

The plan was to measure how much time I spent using the two programs, and also compare the images that resulted from work in  Lightroom Classic to that at the end of my Photoshop “touch-up”.   I realize that time spent was not necessarily an accurate measure of the contributions made by each program.  Lightroom’s intuitive workflow allows for substantial improvements in a relatively short period of time, while subtle refinements in Photoshop can often require more prolonged work.   These examples do not include the time that I spend getting the image ready for a full size JPG upload to the cloud (my web site) and reducing the image size for use on my website.

Lightroom vs Photoshop

Three Little Horses

Final Web Image

Let me start with an easy one.  All over Iceland you will notice groups of small horses seeming to be totally oblivious to the wind and cold.  In years past these cute guys were beasts of burden, but now, I am assured, their only job is to look cute at the side of the road and attract tourist to stop for a petting party.  We found this trio shortly after our visit to the Geysir hot springs.  All adorable, but my favorite shot came when one found the need to comment.

Lightroom Edit

Lightroom: This was an easy edit.  I adjusted the whites and highlights to retrieve some detail in the snow, while setting the black point.  I boosted vibrance slightly and added a bit of texture and clarity.  I usually save cropping for later in Photoshop, but here it was obvious that I had to get rid of the elbow protruding on the left. A bit more work than usual : 15:00 minutes

Photoshop Edit

Photoshop: This is one of those situations where there was little to add with Photoshop.  I used my favorite Shadow/Highlight Tool to add a bit more detail to the wind scoured snowy ground.  I darkened and brought out the color of the hair.  Finally, I drew the eye to the ”laughing” horse with a subtle splash of light.  And I was done.  I normally do a little sharpening in Lightroom, but I don’t sharpen in Photoshop until I am ready to output the image for a specific size and purpose.  Sounds like a lot but only 7:50 minutes

Total :  22:50 Minutes

Gullfoss Falls

One of Iceland’s most dramatic waterfalls is the Gullfoss Falls.  It was truly breathtaking, but what also robbed me of breath was the hurricane force wind that was screaming up the Hvita River Gorge.   I would have loved to spend time exploring all the different angles of this amazing spot, but given the conditions, I could only find two spots that allowed me to stand with a desperate margin of safety. How I avoided being blown away is a story for another time.  Back at home I tried to make the best of what I got.

Lightroom Edit

Lightroom:  My first concern was to avoid blowing out the highlights in the snow and water, especially in the waterfalls.  I dropped down on the highlight slider while watching to avoid making the foreground snow appear a muddy gray.  While setting the dark tones I was careful to avoid losing detail in the rocks. I added a little Texture, Clarity, Vibrance and a touch of sharpening.   As usual, I corrected the Chromatic Aberration.  I reserve cropping for later. 6:12 minutes

Photoshop Edit

Photoshop:  In Photoshop, I started by using the arrange tool to compare four versions of this image, side-by-side.  Studying all the images at 100% magnification I was able to find one that was reasonably sharp.  As my camera and I were being violently blown about, it was surprising that any of the images were sharp. Next, I used the Hue/Saturation Tool to bring out the blue in the water and ice.  Finally, I went into the Camera RAW filter to apply a bit more texture and Clarity.  I planned to use a layer mask to apply this effect just to the falls, but in the end, I liked how adjustment brought out the rugged appearance across the image. 11:52 Minutes

Total : 18:02 Minutes

The Results

I did the time comparison on two other images, both shots I took at the Geysir Hot Springs.  For the record, here are the average times for all four:

Lightroom:  13:30

Photoshop: 16:09

And for completeness, the average time spent fine tuning, cropping, and sharpening for output of a high resolution, full size JPG to the cloud and resizing and sharpening of the approx. 400kb web image:

11:15 Minutes

Grand Total: 40:58

Photoshop Edit
Note the detail in the water

Hot Pot, Geysir Hot Springs
Lightroom Edit

Too Much Time?

Leif Erikson
Hallgrimskirkja Cathedral, Reykjavik

Forty minutes for one image, and these were nor especially challenging.  I can spend hours removing a dense net of wires if the underlying image is strong enough.  I know some very good photographers who say that if they can’t finish editing in about 5 minutes, they throw out the image.  I think they may be missing some great shots.  Personally, I know how hard I work to get in the right spot and strive to get the best digit image in the camera.  For me, no amount of time is too much in the service of bringing an image to its full glory.  As Ansel Adams said, the original negative (or now the raw file) is the score and the final print or image is the performance.  I think that these images show that you can create great performances with Lightroom.  The differences in the Photoshop image are subtle, and I should note that I could have performed more refinements in Lightroom, if I didn’t plan to move the image to Photoshop for further work

Reykjavik Iceland

So get going with Lightroom and if you need some place to start, I will be happy to welcome you at my dining room table next week.

For more pictures of Iceland check out my Iceland Gallery.  I will be adding more images over time.

Content Aware Fill , More Photoshop "Magic"

There is no question, learning Adobe’s Photoshop can be an intimidating endeavor.  The program is so imposing and powerful that many are frightened away before even trying.  It took me several books and courses before I felt that I could begin to use at a small part of this amazing tool.  The fact is that Photoshop is so deep that, I suspect that no one person can fully understand it all.

Chesterfield Donkey

My approach has always been to decide what I wanted to do for my images, and then discover how to use the parts of Photoshop which would help accomplish that goal.  I didn’t worry about everything that Photoshop can do, just those things that I can use.

Of course, when starting, there are a few basic tools that are important to understand.  Importing images, layers, selections and the basic image controls are essential, but what about all that other stuff.  You can go a long way without having to worry about, HDR, panoramic merging, and split toning, but it’s nice to know that all these tools are there just waiting for you to explore – when needed.

One of these tools, which is fairly new, is Content Aware Fill.  Like many of the new tools, Content Aware Fill (CAF) just provides an easier way to do something that you could do before.  It has always been possible to use the Cloning Tool to fill unwanted areas of an image with pixels taken from other parts of the image, or even from another image.  This can be arduous, as you try to find areas that blend seamlessly with the underlying content and texture.  The cloning brush is still an essential tool, but given the right situation, Content Aware Fill can use the amazing computational power of Photoshop to automate a large portion of the job.

Content Aware Fill

In the Edit drop-down menu, Photoshop provides a list of options to fill an area of an image.  A selection can be filled with the current foreground or background color, a pattern or any color you choose.  This is also where the content aware tool is found.  After creating a selection around an unwanted area of the image (eg my face), Content Aware Fill will attempt to fill the area with pixels from around the selection.  It works to blend the fill with the surroundings.  It doesn’t always work perfectly, but often the results are almost magical.  Typically, the filled area can benefit from a bit of touching up with the Cloning Brush or the Healing Brush. I thought of this as I was working on a picture of a pasture lane in Chesterfield.

Pasture Lane

The Leaning Lane

Recently, I was visiting a friend’s farm down a backroad in Chesterfield New Hampshire.  In addition to becoming reacquainted with their semi-friendly donkey, I got a nice shot looking up the pasture lane which reminded me of the power of the Content Aware Tool.  The picture had all the essentials, beautiful light, a nice fence and road all leading to the border tree and even a distant tractor.  Great! But when I got home, I realized that all of my focus stacked images were significantly rotated clockwise. I could just hear the donkey laughing, but, hey, this is digital, all I had to do was rotate the image back to plumb.  A 5.5 deg counter-clockwise rotation brought everything back to normal, but the process left me with four triangular areas of vacant space at each of the corners.

Curing the Triangular Transparencies

My choices were simple, I could either crop out the areas of transparency, use cloning to fill the spaces or do some combination of the two.  I wanted to keep most of the full image, so cloning was the obvious solution.  All of this could be achieved with careful use of the cloning brush, but it was the perfect situation to start with Content Aware Fill.

Fill Menu

I started working on a fresh pixel layer to preserve the original layer and then by selected one of the spaces.  On this picture, selection was easy to do.  On these geometric shapes, I find that the Polygonal Lasso Tool works well.  It is generally good to expand the selection slightly into the surrounding pixels.  I typically use “Modify” “Expand” adding about 5 to 15 Pixels, depending on the size of the selection.    I then pulled up the Content Aware Fill option under the Edit drop-down menu.  

Filled Triangle

The Content Aware Tool has a few options.  “Color Adaptation” should be checked to improve the color blending of the patch.  Since I was filling an area of transparency, the most important thing in this case is to leave the “Preserve Transparency” box unchecked.  Otherwise none of the area will be filled.   I hit “Ok” and waited for the magic. 

Duplicate Fence Post

 In this case the results were generally quite good requiring only a little cloning touch-ups, but the CAF tool works randomly each time and, if you don’t like the results you can always try again. 

As you review the results of the fill, it is important to look for areas where surrounding detail has been noticeably copied into the patch.  This can be seen in the upper left where a fence pole was clearly duplicated.  A little cloning easily removed the obvious defect, but sometime the duplications can be more subtle.  A careful examination is always a good idea.

Madame Sherri’s Hat

Madame Sherri at the Ball 1920s

Like most of Photoshop’s new “magic” tools CAF is not quite as magic as it is suggested in Adobe’s press releases.   CAF does not work in all situations and almost always requires some amount of clean-up to reach the best result.  An example of this can be seen in my article about repairing antique photographs.  The absent area of curtain was first filled with CAF, but the folds did not align with the rest of the material.  With the cloning brush I was able to smooth out the lines to make a more even match.  We can see that Madame Sherri really knew her hats!

Content Aware Fill - Poor Alignment

CAF and Cloned Touch-up

Parrish Shoes

The fictitious Parrish Shoes sign was painted on a wall in downtown Keene NH for the filming of the movie “Jumangi”.  Since then, the town has elected to keep, and regularly refresh, the sign as a local attraction. 




I recently shot the spot from the perspective of the shoes, but of course I couldn’t avoid at least one person waiting at the crosswalk. With the woman against the brick wall, CAF didn’t do a great job.  The brick kept bleeding into the patch.  I tried it a few times, but eventually found it easier to use the Cloning brush.

Bleeding with Content Aware Filll

That’s it.  One of the confusing things about Photoshop is that there is almost always more than one way to achieve the same result.  The challenge is to choose the right tool or tools for the situation.  As you use the CAF tool you will begin to recognize situations where it will work well and other where the surrounding detail may get blended uncomfortably with the result. In these situations, “Content Aware Patch” can provide a bit more control over the sampling area.  But that is a subject for another article and, of course, you can always go back to the good old Cloning Brush.

Jeffrey Newcomer

www.partridgebrookreflections.com