Best Practices for Module Developement and Building

[Jaykul]

Hey, anyone interested in trading success stories about how you "build" modules?

I just tried something new this week on the SecretServer module, thanks to @bushe and @RamblingCookieMonster ... where the functions in the module are organized in "Public" and "Private" folders, and the psm1 dot-sources them, but the build script combines them, copying all the content into the psm1 -- so when it's shipped, the module is just the .psd1 and the .psm1

The result is somewhat easier for code navigation and debugging during dev (at least in Visual Studio, Code, and Sublime) and faster loading of the 'built' module.

I like it so much, I'm wondering if it's worth teaching others to do the same...

The build script for that is something that's gone through many revisions on other projects, and I'm starting to wonder if there's a way we can ever stop all the projects on github from having their own unique build/test systems.

See Also: ModuleBuilder, PSake, PSDeploy, CodeFlow, Pester, etc...

[gerane]

I like this idea a lot. I am currently using public and private folders, but really like the idea of adding them in add build.

[gerane]

I have a plaster template written up using the format I have now and was
going to see if it could be added into the examples.

Edit: Reading comprehension fail.

I have been thinking about the Tests folder as well. Should it be in Root\Tests or Root\ModuleName\Tests

[gerane]

What I encounter is issues with modules using different Tests directories, making reusable tests usually require some sort of alteration.

[zloeber]

I'd love to see how you are doing releases. I've slowly been working on an invoke-build script for my handful of module projects to help turn the source into modules in a more streamlined manner. The invoke-build script is an exceptional, pure posh, build engine which you might appreciate. I run code formatting against the source among other things. Here is the list of tasks I've created so far. I'm not completely done with the build script but the overarching steps are:

• Clean/Create the build temp directory
• In the temp directory:

1. Create project folder structure
2. Copy over project files
3. Format PowerShell files with the FormatPowerShellCode module
4. Create module markdown files (PlatyPS)
5. Convert markdown files to HTML documents for online help (PlatyPS)
6. Update module manifest with exported functions and new version information.
7. Update the current release directory with the temp directory contents

• Create a zipped copy of the current release directory contents with the current version name (thus keeping an ongoing history of releases).
• Push the release with a tag to GitHub
• Push the release to PSGallery (using some custom functions)

I hope to have a working (releasable) .build.ps1 script as an example done soon. I'm sure someone like yourself could take and make it 10x better.

[gerane]

I think it would be interesting to create a PSDeploy deployment type that handles a module like this. I really like PSDeploy DSL for configuration. It lets you extract away the complicated and messy code in your build script, and replace it with a single command. You just have human readable config file. It makes for easier auditing and reading of what is going on.

[gerane]

This could also allow for variations in folder structure. The config file could have paths for public\private\tests etc. Why I mention PSDeploy is it also covers more than just building the module, so while it could have this integrated in, it can also cover things like build PlatyPS help, cloud services or any other sort of deployment.

[bushe]

I am currently using https://github.com/martin9700/ConvertTo-Module to build modules, I have a partially working DSL wrapped around it to make it a simple process, using .Build.ps1 file and just running Invoke-Build.

It might make more sense to have ConvertTo-Module plug into PSDeploy as mentioned by @gerane

[rkeithhill]

For "build" tasks, I prefer PSake. It seems to have reasonable momentum (16 releases, 47 contributors) behind it and it works well. Like the old sayings go: "use the right tool for the job" and "don't try to fit a square peg into a round hole". :-)

[Jaykul]

@bushe yeah, I did not like that one. It's adding all this weirdness about #publish blocks and stuff. Since during development the psm1 is just dot-sourcing all the scripts, there's no need to do any weird magic at build time: just copy the file content. It's literally a one-liner.

I want to keep my dev psm1, and put the release one in the output directory.

And finally, it wanted to re-create the psd1 every time with all the lame "in case you don't know how modules work" comments -- blowing away my comments, and making the PSD1 UTF-16 encoded. That final bit made me so angry, I deleted it. 😠

@gerane I'm increasingly confused by PSDeploy vs. PSake and what the goal is 😉

[zloeber]

I went ahead and finished the invoke-build script I've been working on for my script formatting module and pushed it up if anyone is interested in it. The build process is pretty simple to run (the last two lines are the different defined task lists ( . = default task if no tasks are given to invoke-build) but you can call any one task individually. I'm no build expert but now that I have it working I'm absolutely going to be using this for all my future projects :)

[zloeber]

I should be honest and say that I didn't actually finish the tasks for pushing to the psgallary (just the underlying functions I'm going to use within them are done, I'll publish them soon)

[Jaykul]

It looks like I need a whole conversation about "build systems" ...

PSake vs PSDeploy vs Invoke-Build.

MSBuild vs Grunt.

Why is everyone trying to create these non-deterministic "task" systems? What do you see as the payoff, versus a simple, linear build script?

[zloeber]

I personally was looking for a completely stand-alone (portable) method for repeating my build releases. I was going to just script out the process but putting the whole thing into the invoke-build framework allows a lot more flexibility for running the individual tasks. If you look at my small(ish) example I have one big linear task list as my default build action but I also have a task called 'TestModule' to kick off a test run of the code formatting module instead of a full release. It also forces you to break your build tasks out into discrete tasks I suppose.

[nightroman]

@Jaykul

Why is everyone trying to create these non-deterministic "task" systems? What do you see as the payoff, versus a simple, linear build script?

I tried to answer this question: Invoke-Build/wiki/Concepts.

[Jaykul]

@nightroman but your tasks don't have "inputs" (source) and "outputs" (destination) -- so doing two builds in a row does all the work twice?