Direct Access 2012 and Outlook RPC/HTTP

I have a client for whom i recently implemented 2012 Direct Access…. there was a number of teething problems in the implementation – but once its working, DA is actually pretty good.

If i could ask the product team for anything in DA 2012 R2, it would be

  • re-introduce wildcard certificate support (it was in UAG DA for 2008 R2, not sure why it got left out of 2012)
  • Make the status reporting dashboard check for local firewall status (that its on and that the rules have been correctly added) as part of its health check
  • Release Win7 SP2 and make it support “simple” DA auth (i know this isn’t going to happen… but it doesnt mean it wouldn’t be nice!)

Anyhoo….. at this particular client, they had decided to do one of my most hated things – use the same url internally and externally for the configuration of web services in exchange. I avoid this configuration where-ever possible…. although i understand at small sites where only one server is present – it may seem like the only option – solutions such as TMG (or another reverse publsiher), SAN certificates or creating additional exchange web services on different ports are all valid ways of getting around that config.

As to why it had been done at this client who is substantially larger…. well, the consultancy that did it…. i have very different views to them on good practices for exchange design…. and the client has a DAG without a load balanced CAS – so there is no true fail-over anyway…. grrr…. but i digress.

Due to the configuration of the same url’s internally and externally for web services, i added an entry into the DA NRPT so it would always resolve to the internal name…. issue is, that users were still getting prompted for auth, because the outlook client detected a slow connection and used RPC/HTTP, which is configured for basic auth (another config item i diagree with).

What i had not know before (because i hadnt needed to) was that you can disable RPC/HTTP on a per-mailbox basis…. using the command

Set-CASMailbox –Identity <alias> –MAPIBlockOutlookRpcHttp $True

While the better solution is to fix the exchange design (different URL’s and NTLM for RPC/HTTP) – its a handy work-around.

Exchange 2013 Gotcha’s

Found this article today

http://theessentialexchange.com/blogs/michael/archive/2013/01/06/exchange-server-2013-gotchas.aspx

it’s nigh on 3 months old – but it lists some of the issues with Exchange 2013 RTM.

Prior to finding this article – I hadn’t been able to find any list of issues with the release, but my own list of issues was growing.

Exchange 2013 release seems to have been rushed a little, but lets hope that these issues are addressed by CU1 (which is due anytime now) – and I’m really hoping the EMC makes it back in by SP1 (but I doubt it!)

Exchange 2010 recovery scenario – OS disk dead

Exchange recoveries don’t come up very often these days (for me at least), with exchange 2010 and win 2008R2 being so solid… and virtualisation… the loss of server hardware or corruption of the OS is just becoming less and less common.

Anyhoo – a client had a SAN disk die – and for reasons which I didn’t delve too deeply into, apparently the RAID5 didn’t work and they lost one vdisk…. (yes, it sounds suss to me too…. this was a EVA4400… so not a cheap SAN)  this vdisk happened to have the exchange OS partition on it…. but the database and log disks were fine….

So, we built up a server with the same name as the old one, gave it the same IP and joined it the domain…. the connected the logs and database drives to the newly built server.

Using ADSIEdit, I then looked up the path to the database, as the client could not remember which drive had which letter – if your in this situation, fire up ADSIEdit and connect to the configuration partition, then navigate to CN=DatabaseName,CN=Databases,CN=Exchange Administrative Group (FYDIBOHF23SPDLT),CN=Administrative Groups,CN=ORGNAME,CN=Microsoft Exchange,CN=Services,CN=Configuration,DC=blah,DC=au

have a look at the properties msExchEDBFile and msExchESEParamLogFilePath…. this will help you allocate the drive letters correctly should you run into a site that doesn’t know this or have it documented.

Next issue – versioning…. the client wasn’t sure which Service Pack version exchange 2010 was at, but though it was SP2.

I grabbed the SP2 media and after installing the pre-reqs, ran setup /mode:RecoverServer

First error came up for the CAS and HT roles…. must install a version of 14.1.218.5 or later….. that number relates to Exchange 2010 SP1 RTM…. funny, SP2 is later than SP1… but oh well…. SP1 it is….

So I grabbed the SP1 media and re-ran the setup command…. and proceeded to get

[ERROR] The internal transport certificate for the local server was damaged or missing in Active Directory. The problem has been fixed. However, if you have existing Edge Subscriptions, you must subscribe all Edge Transport servers again by using the New-EdgeSubscription cmdlet in the Shell

because there is not another exchange server within this environment, I cant simply remove the edge transport from the GUI… so I went searching in ADSIEdit to

CN=HTServer,CN=Servers,CN=Exchange Administrative Group (FYDIBOHF23SPDLT),CN=Administrative Groups,CN=ORGNAME,CN=Microsoft Exchange,CN=Services,CN=Configuration,DC=blah,DC=au

Go to the properties of the HTserver container and the properties of the property “msExchEdgeSyncCredential” – remove the 3 entries.

Try the recovery again, and get

     A Setup failure previously occurred while installing the HubTransport role.

Either run Setup again for just this role, or remove the role using Control Panel.

Awesome…. try to uninstall via control panel, no go, try to uninstall via command line, no go…. so, the solution:

  • Open up regedit
  • Navigate to HKLMSoftwareMicrosoftExchangeServerV14
  • Delete the “Action” and “watermark” keys

Now re-run the install – all is good…. yay!

Next up, re-allocate the certificates on the server, re-associate the edge transport – and test (and then upgrade to SP2 RU5v2 as of time of writing!)

All in all, the recovery procedure is still good, but complicated a bit by the edge transport being present (im not a fan of edge transports…. use a HT with forefront for exchange with port 25 published via TMG instead….. or if you prefer, an ironport)

Exchange/Lync/Office 2013 now available

well, so much for “available to technet subscribers in mid november” – Exchange, Lync and Office 2013 are available on the technet downloads site now….

Exchange 2013 is kind of usless right now – as to interoperate with exchange 2010, SP3 is required (http://technet.microsoft.com/en-us/library/aa996719(v=exchg.150).aspx) – and SP3 isnt out yet – and according to this (http://blogs.technet.com/b/exchange/archive/2012/09/25/announcing-exchange-2010-service-pack-3.aspx) wont be out until “1st half” of 2013…. thats a potential long wait…. so really, the product has bene released before its ready.

Microsoft and management interfaces….

Consistency…. its what we spend a great deal of time fixing for clients…. sure, they see us writing standardisation documents for AD/Exchange/Group Policy/Logon scripts etc….. but really, what we are doing is trying to delivery consistency – simply because the more things that are consistent within an environment, the more logical it is – and the easier it is to support.

Take for example the very simple case of mapped drives….. some clients have the whacky situation where S: is mapped to \server1common for some people and \server7finance for others…. this ofcourse makes the job the the helpdesk much harder – as users dont know (nor should they need to) what a unc is…. they just cant find the file on their S: drive. A consistent naming standard for user accounts, email addresses etc are commonly used by the majority of places we deal with – as it just makes sense.

Anyhoo – onto the point of the post – Microsoft products and their management interfaces.

A while ago, the NT 4 option pack to be exact, MMC was introduced, with talk about MS products standardising on using MMC’s for management…. this kind of took off…. AD tools, computer management, event viewer, services etc etc all were (and still are) available as mmc snap-ins….. very handy for standardised management and also creating your own snap in. Other products such as Exchange, ISA/TMG, OCS and SCCM used an MMC-like management console….

Some of these weren’t necessarily the best….SCCM 2007 was universally panned for its shit interface (but it still did the job)…. OCS 2007 i didnt think was that bad, but the devs apparently did…. and exchange 2010 EMC was fucking awesome (a bit slow… but functionality and layout wise it was great)

On the upside, the SCCM 2012 interface is not an MMC at all, and its awesome… well laid out, relatively easy to find stuff, responsive etc….

The we have Lync 2010 with the silverlight interface…. for some of the admin tasks, but an MMC for the topology tasks and powershell for all taks, but some must be completed via powershell. While i find it usable – and dont mind it, its a bit dis-jointed.

Then we have the slight shift to the side – in powershell. Basically the only was to do all the tasks required in exchange 2007/2010 and Lync was to use powershell…. this was a little annoying for clients, but offered an absolute crapload of scripting power… so it was, on the whole, awesome.

Now, we have the wave 15….. SCCM 2012 interface is awesome…. Lync 2013 interface is the same a 2010…. a bit of a (oddly ok) thrown together mees…. Exchange 2013 has thrown out the awesome 2010 EMC and replaced it with a web interface, which is ok… but not as good as the interface it replaced!

So, whats the point of this article… give us some consistency!

1) Having a powershell interface for products is awesome…. keep doing it – and keep exapnding the number of features it covers. The commandlets however need to keep consistency as well… which so far, has been reasonable (i have a vague memory of a few of the Lync commandlets not quite fitting wtih the get/set/new/remove nomenclature)

2) A web interface for certain admin tasks as an option is fine – but a web interface should never be the primary interface in my opinion…. sure the silverlight ones are slightly less painful – but they still suck compared to a GUI

3) Give us a consistent GUI interface…. i get that the teams internally at MS might not see eye to eye a lot of the time and that the interface has to be right for the product…. but FFS… the reason that the OCS 2007 interface sucked, wasnt because mmc sucks, its because the OCS interface sucked. The SCCM 2012 interface is awesome – and even though its not an MMC, I think that same type of interface would also fit well for exchange/lync/AD etc….

4) i’d be disappointed if MS went all web with every interface, but if they did it for every product – at least it would be consistent!

In short MS – please give us some consistency with your management interfaces across product lines….

August 15th – Win 2012 and Win 8 available for DL

http://www.bink.nu/windows-8-has-reached-the-rtm-milestone

I’ve made the (fair) assumption that server will become available at the same time as the client.

Im not convinced that many people (in the corporate space) actually care about Win 8…. part of that is because Win7 is a bloody good OS…. if Win-7-to-go existed… i think there would be even less interest in Win 8.

Server 2012 is a bit of a different matter…. while many features are a bit ho-hum, Hyper-V replica’s… there is some big cash savings to be made by swapping from VMWare and SRM… so will be interesting to see how that plays out.

Oh – and the rest of the Wave 15 products that have previews – Exchange, Lync and office 2013… well, im also quite “meh” about them too…. dont get me wrong – its not as if they are terrible – but there is just nothing particuarly exciting about any of them.

The exchange 2013 web management interface, after a few days of using it, i no longer dislike…. i fucking hate it…. moving to that after using the very good exchange 2010 management console is a huge step backwards…. i can only assume the manager that made that decision had smoked something or was busy crapping on about how great “the cloud” was to some sales idiot… or both.

With Lync, its cool to have the web client back – outside of that, while there are improvements, im a bit meh….. annoyed at the lack of authenticated sip trunk – still.

Office…. well, its office… i can imagine its hard for those guys to come up with anything new… it would be nice if they worked with the exchange team to achieve true, no impact exchange failovers… (which they well may have, but im not setting up test DAG’s and CAS arrays to find out until RTM)

On the up side, the more i use SCCM 2012 – the more it rocks. Sure, there are still improvements that could be made….. but holy shit it has come a long way since 2007, its so much more usable, responsive and quick compared to 2007. Now theres a team that got their shit together – well done.

Exchange and Lync 2013 – a few more details…

http://www.zdnet.com/microsofts-exchange-server-2013-whats-new-7000001407/

http://technet.microsoft.com/en-us/library/gg398616(v=ocs.15)

http://ucken.blogspot.com.au/2012/07/whats-new-in-lync-2013.html

First thing that caught my eye – “Replacement of the Exchange Management Console by a Web-based Exchange Administrative Center (EAC)”

FFS. Many MS products seem to flip flop from web-based admin to MMC and back again. I fucking hate web-based admin. Sure as an additional administration option, web based is great…. but the primary admin tool should always be an MMC or MMC like interface and powershell (or some type of command line interface)…. the decision to go back to web based is, IMO, extremely disappointing. The responiveness and usability of web-based is fundamentally flawed byt the very nature of what it is.

Next up- ” Better integration with Lync and sharepoint” – this could be great….. but define “better”… each version always advertises tighter integration…. (as they should) – but until i have some type of specifics – this tells me nothing.

The Lync 2013 post lists a bunch of new feastures, many of which are useful – but none that are excessively exciting. It appears that there is still no option for using an authenticated sip trunk – which is very disappointing.

I am currently having a play around with these preview versions in a virtual environment – and im not a fan of the management so far.. (exchange 2010 EMC was great – why replace it?) – anyhoo…. as i use it more, obviously my opinion may change… 🙂

SenderBase reputation is poor…

This pearler popped up today.

We started getting a few delays messages from various clients…. at first didnt think much of it…. but after telnetting into one clients and recieving the following error:

554 Your access to this mail system has been rejected due to the sending MTA’s poor reputation. If you believe that this failure is in error, please contact the  intended recipient via alternate means.

Connection to host lost.

Thats not so good… mxtoolbox.com…. nup, not on those 107 blacklists, abuse.net, yep, we’re still not an open relay, nor have we ever been… so WTF?

http://www.senderbase.org/help/blocked

apparently our subnets reputation is poor….. not our relays or our /28, but someone else in the class C… thats right, the fucking class C.

I had always thought that ironport was quite good – up until now….. thats just plain fucking stupid…. punishing the entire class C because one dickhead is being a dickhead….. effectively shutting down our business for a few days.

Best thing is – there’s no appeal or support line to contact… best article i found was this – http://itepisodes.blogspot.com.au/2009/04/what-to-do-after-being-blacklisted.html that suggestes to wait 3 days and allow the reputation to come back up (assuming the SPAM stops)

* Update 4/4/2012 @ 17:23 ACST *

A guy i pseudo work with every now and again, who specifically does cisco and ironport stuff rang me out of the blue last night and said he was also having the issue, along with mqny others (and i got a bunch more calls today about it too) – looks like the issue is senderbase…. the guy managed to get this out of them….

“While investigating the IP in question, SenderBase identified a misconfiguration in one of their sensors which was causing the discrepancy with the IP. This has been fixed and steps have been taken to ensure it does not happen again.   The reputation of the IP should improve within 24 hours as our servers update the changes made on our end.”

So while that will hopefully resolve the issue – im still pissed. For a few reasons:

1) Im not quite sure these people understand the impact to a company of not being able to contact their clients via email

2) Go to http://www.senderbase.org/help/blocked – what options do you have… you can query your reputation and thats it. Sure you can hit contact, then email support…. but not having an automated mechanism for logging de-listing, come across, to me, as if “we never get it wrong, therefore if your network has a poor reputation, you have clearly fucked up” – when thats not the case.

3) The guy who forwarded me this email is a fairly significant cisco dealer (for this region) – and a good dude. If he has better contacts than us non-cisco people and still has to jump up and down for 3 days to get a response…. thats pretty fucking shit. See point 1 – i just dont think they realise, or care, about the impact that it has on a business.

4) Fair call to the guy that commented (below) – no i dont know the other site is spamming…. i jumped to an incorrect conclusion – and i was completely wrong

As of this moment, we are still blocked due to “poor reputation”…. we’ll see if anything changes overnight.

* Update 5/04/2012 @ 08:47 ACST *

Still no movement, i have directly emailed senderbase support – i dont expect a reply – but have to try…. we now havent been able to email a good 60% of our customers all week.

Brett – the commenter below has also posted an article about this on ITNews.com.au – http://www.itnews.com.au/News/296066,having-trouble-sending-email.aspx

* Update 13:40 ACST *

So i just had another chat to the guy i know that does a lot of ironport stuff…. apparently something is in the works. I have asked him to put together some facts around the situation and send them to me – which he said he will try and do this afternoon.

Since this doesnt look like its going to be fixed soon (enough) – i have implemented a temporary work around on my mail servers, of relaying via my ISP’s mail relay…. for those of of you out there also on internode…. you can use mail.internode.on.net as a outbound relay…. at least temporarily.

On the reverse lookup/PTR comments below…. bascially there was some suggestion that it was a stricter PTR check that was causing the issue initially… i got this information 3rd hand and ran with it – as, no official information was forth-coming from cisco…. so hey, trying something on a whim that may fix the issue is good by me – as it can always be reversed easily 🙂

Anyhoo – hopefully we should have something better by the end of today.

* update 9/4/2012 * – i did get this thurs night, but have been away camping, so didnt post.

http://www.itnews.com.au/News/296227,cisco-says-mea-culpa-on-bounced-emails.aspx

I can now also report that i can connect directly to all of our clients.