Appeal for advice: tying together Windows and Linux automation

Date October 15, 2012

I love the smell of new machines in the morning.

I'm automating a machine creation workflow that involves several separate systems across my infrastructure, some of which involve 15 year old perl scripts on Solaris hosts, PXE Booting Linux systems, and Powershell on Windows Server 2008.

I can script each of the individual parts, and integrating the Linux and Unix automation is fairly straightforward, but I'm at a loss as to how to reliably tie together the Powershell scripts to the rest of the processes.

I would prefer if the process began on a Linux host, since I imagine that it will end up as a web application living on an Apache server, but if it needs to begin on Windows, I am hesitantly okay with that.

I would ideally like something along the lines of psexec for Linux to run against Windows, but the answer in that direction appears to by Cygwin, and as much as I appreciate all of the hard work that they put in, it has never felt right, if you know what I mean. It's great for a desktop and gives a lot of functionality, but I feel like Windows servers should be treated like Windows servers and not bastardized Unix machines (which, incidentally, is my argument against OSX servers, too, and they're actually Unix). Anyway, I don't want to go with Cygwin unless that's the last and only option.

So I guess what I'm asking is if you know of a way to execute jobs on Windows machines from Linux. Without Cygwin. I'm open to ideas and suggestions, including "Look idiot, everyone uses Cygwin, so suck it up and deal with it". Thanks in advance!

  • Dustin J. Mitchell

    I'd avoid Cygwin if at all possible. It has a lot of unpredictable failure modes that will make your life much more of a living hell than Windows.

    At Mozilla, we're planning to use Puppet, actually. Our linux and OS X config management is 100% puppet after base system install; for Windows, we'll be using WDS/MDT for base system plus some cranky apps like VS2010; GPO for domain-wide policy enforcement and Windows-specific tasks; and Puppet for everything else, particularly things that need to be the same across our Windows and *nix hosts, like NRPE and SSH configurations.

    Anyway, puppet can do registry entries, control services, create files (albeit not with full management of the NTFS permissions), and - what you want - run executables. PuppetLabs has a nice quiet-install MSI.

    Disclaimer: I haven't actually *implemented* any of this yet, but I'm cautiously optimistic.

  • James

    Have you looked into a scheduler application such as control-m?

  • http://www.standalone-sysadmin.com Matt Simmons

    Dustin: wow, nice! And it looks like mcollective runs against it, too! Hrm. That's very, very tempting. I'll consider. Thanks!

  • http://www.standalone-sysadmin.com Matt Simmons

    James: I haven't. What's the ballpark price point on something like that?

  • http://www.squiggle.org Geoff

    Crude method....

    - Writeable fileshare on the windows box (secured!)
    - Scheduled task running on windows box that runs a powershell script every few mins
    - Powershell looks for .cmd files in the fileshare, executes them, then deletes the .cmd file

  • John

    I have not used it (yet), but would FOG (http://www.fogproject.org/?q=node/1) and puppet work for you?

    I don't know how enterprise ready FOG is, but their overview states that they can deploy through PXE and Tftp.

  • http://saintaardvarkthecarpeted.com/blog Saint Aardvark

    WPKG might work...

  • Dustin J. Mitchell

    Fog's not bad, but it's not great, either. For the same price (free), you can use WDS/MDT, and get a lot more flexibility in setting up new machines.

  • Prashanth

    I agree 100% that windows & Unix systems should be managed seperately.

    We use Windows SCCM to deploy and manage windows systems. There is a learning curve but once you are thru it, it just a breeze. SCCM basically integrates MDT, WSUS, AD, GPO etc to form a unified console and makes it more powerful.

    For Example: My windows 7 deployment project was just as breeze as PXE booting a computer and in 45mins it is ready to go. You can migrate XP users and their files and deploy updates pkgs just like linux. I do not maintain any captured images but simply use the bases .wim files and layer it with silent install of applications via SCCM.

    There is PSEXEC tool integrated(Right-click tools), so you can execute commands remotely and run same command on all systems via TaskSequence.

    There is so much more, but I will leave that to you to investigate.

    Best
    Prashanth

  • James

    Ballpark?
    80 k
    However you are purchasing a product that sets the standard.
    If you are looking for something much easier on the wallet look at job2Do

  • http://meine-er.de Holger

    Similarly to Puppet, Chef also has rather good support for Windows nowadays (including running batch and powershell scripts, editing the registry, installing various package types, creating services, ...) You an even fully bootstrap Windows boxes. And it allows you to centrally manage all your UNIX and Windows boxes using either a self hosted server or Opscode's SaaS offering.

    It just depends on which you like more, Puppet or Chef :)
    (Disclaimer: I haven't used it with Windows yet, but do like the UNIX part.)

  • http://www.chriscowley.me.uk Chris Cowley

    I have used WinEXE before (http://winexe.sourceforge.net/)

  • Paul Tötterman

    Like others have said, there's WinEXE. I use puppet for what I can (installing packages, managing files and registry) and launch powershell scripts through puppet when I need. But I really should just pay for the appropriate windows server and ca licenses and use ad & gpo.

  • http://ninet.org Ryan McLean

    I'm just testing Puppet & CFEngine for our linux management although I have read ey can manage windows to some extent. I come from a heavy SCCM backround and SCCM 2012 can (allegedly) manage linux systems, so it might be worth investigating if you have access to MSDN o you have System Center Licensing.

    IMHO SCCM is the best CM tool for windows on the market, as this is my 1st attempt at CM on Linux I can't really comment to much but would be interested in seeing what solution you end going for & why.

  • http://rejectreality.net Greg

    Have you considered abusing Continuous Integration software as a cross-platform orchestration tool?

    Install the CI master wherever is most convenient, install the agent on your Windows box (either this or this), configure a job to execute your powershell script (either directly invoking it using the Windows Batch Command config, or using a plugin if you want to write/keep your script inside the CI app) on your Windows agent and trigger the job remotely via curl or similar.

    (Response crossposted to your ServerFault question)

  • Dustin J. Mitchell

    Remember Matt asked about running scripts, not CM. But also bear in mind that "CM" means different things in the Windows and UNIX worlds. For Windows, it generally means starting with a full Windows installation, adding third-party software and disabling or restricting access to Windows features; for UNIX, it means fully specifying an system's configuration in a way that allows the configuration to be changed after the fact without reboots or reinstalls, and with appropriate levels of abstraction to be able to customize huge numbers of machines with very little code.

    SCCM's really good. Puppet's really good. But they don't do the same thing.

  • joe

    or Just install a ssh server on the windows box so you can remotely log in and run whaterver you want. don't think it needs cygwin.

  • Dustin J. Mitchell

    There are a few SSH servers for Windows. Those that don't require Cygwin will dump you at a DOS prompt. The one we use does some kind of screen-scraping, so if login via SSH within a few seconds of logging out, you actually see the same screen that you saw before.. which is a little weird. I've never tried logging in twice at the same time.

    My pint is, that might be a workable debugging technique, but it's not a good choice for any kind of automation.

  • http://www.ewanleith.com Ewan

    Have a look at PowerShell Server

    http://www.powershellinside.com/powershell/ssh/

    It lets you run powershell commands on a Windows box via SSH from your Linux server. Works well, and there's a (limited) free version.

  • Paul Tötterman

    There's also FreeSSHd ( http://www.freesshd.com/ )

  • http://ninet.org Ryan McLean

    As you are trying to achieve a workflow perhaps system center orchestrator might be of use coupled with http://www.christowles.com/2011/06/how-to-ssh-from-powershell-using.html

    Might mean you need to start from windows rather that linux though.

  • http://blog.oneiroi.co.uk David

    I've started exploring puppet here myself for rapid prototyping, and ongoing management for concept -> development -> Q&A -> production including in the majority of cases rebuilds throughout the process (IaaS + PaaS), whilst I have a vehement hate for anything windows I can understand people having to work with it.

    And as you can use case statements within your puppet manifests based on the global var $operating system I really do recommend using puppet if you absolutely have to work with windows, you can therefor maintain modules + manifests and configurations for "common" systems, despite variance in the os.

    If like me you are paranoid and want to author everything yourself, bare in mind writing your modules and manifests will take a significant investment of time, however the subsequent maintenance of said configurations should be low.

    Whilst I've not used puppet to configure windows it does seem possible: http://projects.puppetlabs.com/projects/1/wiki/Puppet_Windows

    Thus maintaining windows as windows with the small exception of a client that checks in with your puppet master running on linux for configuration :)

  • David F

    I haven't tried it, but what about running psexec under Wine?

  • http://blog.usepowershell.com Steven Murawski

    How about OMI (formerly NanoWBEM), which is a CIM server for a variety of platforms. This gives you WS-Management support (which is what PowerShell Remoting and CIM in the Windows Management Framework 3 use).

    http://blogs.technet.com/b/windowsserver/archive/2012/06/28/open-management-infrastructure.aspx

    https://collaboration.opengroup.org/omi/?gpid=680

  • Dustin J. Mitchell

    CI for configuration management? That's insane!

  • bshomb

    We use Jenkins, right now we run VB Scripts to push msi's silently via wmi, but you can script whatever you want I imagine.

  • darknerd

    WinEXE mentioned above, as it seems to leverage off of SAMBA. SAMBA folks already have RPC automation (Windows RPC is how their tools remotely execute across other systems). Thus, using something like WinEXE would be the most plausible path as you are using what Windows uses from Linux/UNIX.

    SSH solutions are nice, but they are not native to Windows and require installing stuff on Windows.