Back to the Loopback: Troubleshooting Group Policy loopback processing, Part 2

Welcome back!  Kim Nichols here once again with the much anticipated Part 2 to Circle Back to Loopback.  Thanks for all the comments and feedback on Part 1.  For those of you joining us a little late in the game, you'll want to check out Part 1: Circle Back to Loopback before reading further.

In my first post, the goal was to keep it simple.  Now, we're going to go into a little more detail to help you identify and troubleshoot Group Policy issues related to loopback processing.  If you follow these steps, you should be able to apply what you've learned to any loopback scenario that you may run into (assuming that the environment is healthy and there are no other policy infrastructure issues).

To troubleshoot loopback processing you need to know and understand:

1. The status of the loopback configuration.  Is it enabled, and if so, in which mode?
2. The desired state configuration vs. the actual state configuration of applied policy
3. Which settings from which GPOs are "supposed" to be applied?
4. To whom should the settings apply or not apply?
1. The security filtering requirements when using loopback
2. Is the loopback setting configured in the same GPO or a separate GPO from the user settings?
3. Are the user settings configured in a GPO with computer settings?

What you need to know:

Know if loopback is enabled and in which mode

The first step in troubleshooting loopback is to know that it is enabled.  It seems pretty obvious, I know, but often loopback is enabled by one administrator in one GPO without understanding that the setting will impact all computers that apply the GPO.  This gets back to Part 1 of this blog . . . loopback processing is a computer configuration setting. 

Take a deep cleansing breath and say it again . . . Loopback processing is a computer configuration setting.  :-)

Everyone feels better now, right?  The loopback setting configures a registry value on the computer to which it applies.  The Group Policy engine reads this value and changes how it builds the list of applicable user policies based on the selected loopback mode.

The easiest way to know if loopback might be causing troubles with your policy processing is to collect a GPResult /h from the computer.  Since loopback is a computer configuration setting, you will need to run GPResult from an administrative command prompt.

The good news is that the GPResult output will show you the winning GPO with loopback enabled.  Unfortunately, it does not list all GPOs with loopback configured, just the one with the highest precedence. 

If your OU structure separates users from computers, the GPResult output can also help you find GPOs containing user settings that are linked to computer OUs.  Look for GPOs linked to computer OUs under the Applied GPOs section of the User Details of the GPResult output. 

Below is an example of the output of the GPResult /h command from a Windows Server 2012 member server.  The layout of the report has changed slightly going from Windows Server 2008 to Windows Server 2012, so your results may look different, but the same information is provided by previous versions of the tool.  Notice that the link location includes the Computers OU, but we are in the User Details section of the report.  This is a good indication that we have loopback enabled in a GPO linked in the path of the computer account. 

   
Understand the desired state vs. the actual state

This one also sounds obvious, but in order to troubleshoot you have to know and understand exactly which settings you are expecting to apply to the user.  This is harder than it sounds.  In a lab environment where you control everything, it's pretty easy to keep track of desired configuration.  However, in a production environment with potentially multiple delegated GPO admins, this is much more difficult. 

GPResult gives us the actual state, but if you don't know the desired state at the setting level, then you can't reasonably determine if loopback is configured correctly (meaning you have WMI filters and/or security filtering set properly to achieve your desired configuration). 

     
Review security filtering on GPOs

Once you determine which GPOs or which settings are not applying as expected, then you have a place to start your investigation. 

In our experience here in support, loopback processing issues usually come down to incorrect security filtering, so rule that out first.

This is where things get tricky . . . If you are configuring custom security filtering on your GPOs, loopback can get confusing quickly.  As a general rule, you should try to keep your WMI and security filtering as simple as possible - but ESPECIALLY when loopback is involved.  You may want to consider temporarily unlinking any WMI filters for troubleshooting purposes.  The goal is to ensure the policies you are expecting to apply are actually applying.  Once you determine this, then you can add your WMI filters back into the equation.  A test environment is the best place to do this type of investigation.

Setting up security filtering correctly depends on how you architect your policies:

1. Did you enable loopback in its own GPO or in a GPO with other computer or user settings?
2. Are you combining user settings and computer settings into the same GPO(s) linked to the computer’sOU?

The thing to keep in mind is that if you have what I would call "mixed use" GPOs, then your security filtering has to accommodate all of those uses.  This is only a problem if you remove Authenticated Users from the security filter on the GPO containing the user settings.  If you remove Authenticated Users from the security filter, then you have to think through which settings you are configuring, in which GPOs, to be applied to which computers and users, in which loopback mode....

Ouch.  That's LOTS of thinking!

So, unless that sounds like loads of fun to you, it’s best to keep WMI and security filtering as simple as possible.  I know that you can’t always leave Authenticated Users in place, but try to think of alternative solutions before removing it when loopback is involved. 

Now to the part that everyone always asks about once they realize their current filter is wrong – How the heck should I configure the security filter?!

Security filtering requirements:

1. The computer account must have READandAPPLY permissions to the GPO that contains the loopback configuration setting.
2. If you are configuring user settings in the same GPO as computer settings, then the user and computer accounts will both need READandAPPLY permissions to the GPO since there are portions of the GPO that are applicable to both.
3. If the user settings are in a separate GPO from the loopback configuration setting (#1 above) and any other computer settings (#2 above), then the GPO containing the user settings requires the following permissions:  

Merge mode requirements (Vista+):

Replace mode requirements:

Tools for Troubleshooting

The number one tool for troubleshooting loopback processing is your GPRESULT output and a solid understanding of the security filtering requirements for loopback processing in your GPO architecture (see above).

The GPRESULT will tell you which GPOs applied to the user.  If a specific GPO failed to apply, then you need to review the security filtering on that GPO and verify:

• The user has READ and APPLYpermissions
• Depending on your GPO architecture, the computer may need READor it may need READ and APPLY if you combined computer and user settings in the same GPO.

The same strategy applies if you have mysterious policy settings applying after configuring loopback and you are not sure know why.  Use your GPRESULT output to identify which GPO(s) the policy settings are coming from and then review the security filtering of those GPOs. 

The Group Policy Operational logs from the computer will also tell you which GPOs were discovered and applied, but this is the same information that you will get
from the GPRESULT.

Recommendations for using loopback

After working my fair share of loopback-related cases, I've collected a list of recommendations for using loopback.  This isn’t an official list of "best practices", but rather just some personal recommendations that may make your life easier.  ENJOY!

I'll start with what is fast becoming my mantra: Keep it Simple.  Pretty much all of my recommendations can come back to this point.

1. Don't use loopback  :-) 

OK, I know, not realistic.  How about this . . . Don't use loopback unless you absolutely have to. 

• I say this not because there is something evil about loopback, but rather because loopback complicates how you think about Group Policy processing.  Loopback tends to be configured and then forgotten about until you start seeing unexpected results. 

2. Use a separate GPO for the loopback setting; ONLY include the loopback setting in this GPO, and do not include the user settings.  Name it Loopback-Merge or Loopback-Replace depending on the mode.

• This makes loopback very easy to identify in both the GPMC and in your GPRESULT output.  In the GPMC, you will be able to see where the GPO is linked and the mode without needing to view the settings or details of any GPOS.  Your GPRESULT output will clearly list the loopback policy in the list of applied policies and you will also know the loopback mode, without digging into the report. Using a separate policy also allows you to manage the security of the loopback GPO separately from the security on the GPOs containing the user settings.

3. Avoid custom security filtering if you can help it. 

• Loopback works without a hitch if you leave Authenticated Users in the security filtering of the GPO.  Removing Authenticated Users results in a lot more work for you in the long run and makes troubleshooting undesired behaviors much more complicated.

4. Don't enable loopback in a GPO linked at the domain level!

• This will impact your Domain Controllers.  I wouldn't be including this warning, if I hadn't worked several cases where loopback had been inadvertently applied to Domain Controllers.  Again, there isn’t anything inherently wrong with applying loopback on Domain Controllers.  It is bad, however, when loopback unexpectedly applies to Domain Controllers.
• If you absolutely MUST enable loopback in a GPO linked at the domain level, then block inheritance on your Domain Controllers OU.  If you do this, you will need to link the Default Domain Policy back to the Domain Controllers OU making sure to have the precedence of the Default Domain Controllers policy higher (lower number) than the Domain Policy.
• In general, be careful with all policies linked at the at the domain level.  Yes, it may be "simpler" to manage most policy at the domain level, but it can lead
  to lazy administration practices and make it very easy to forget about the impact of seemingly minor policy changes on your DCs.
• Even if you are editing the security filtering to specific computers, it is still dangerous to have the loopback setting in a GPO linked at the domain level.  What if someone mistakenly modifies the security filtering to "fix" some other issue.
• TEST, TEST, TEST!!!  It’s even more important to test when you are modifying GPOs that impact domain controllers.  Making a change at the domain level that negatively impacts a domain controller can be career altering.  Even if you have to set up a test domain in virtual machines on your own workstation, find a way to test.

5. Always test in a representative environment prior to deploying loopback in production.

• Try to duplicate your production GPOs as closely as possible.  Export/Import is a great way to do this.
• Enabling loopback almost always surfaces some settings that you weren't aware of.  Unless you are diligent about disabling unused portions of GPOs and you perform periodic audits of actual configuration versus documented desired state configuration, there will typically be a few settings that are outside of your desired configuration. 
• Duplicating your production policies in a test environment means you will find these anomalies before you make the changes in production.

That’s all folks!  You are now ready to go forth and conquer all of those loopback policies!

Kim “1.21 Gigawatts!!” Nichols