Introducing Our Latest Innovations: CReW Macros - Ensure Path and ZIP Output

 Greetings, data enthusiasts and Alteryx aficionados! We are thrilled to unveil two exciting additions to our Chaos Reigns Within Alteryx Macro family. Say hello to "Ensure Path" and "ZIP Output," two remarkable macros designed to take your data workflow to the next level.

Available now on Alteryx Marketplace...

Key Take-Away:

These macros were designed to give you more flexibility in automation of your workflows.  They permit you to message, warn and error so that you can take full advantage of the control containers.  

They use Python tool for the execution of the creation of paths and zip output.  As such, we capture error codes returned (if any) in a STATUS message on each macro.  Python takes time to load, so there is a penalty to pay for using the tools.

Ensure Path

Ensuring Smooth Sailing in Your Data Journey

Managing directory paths can be a daunting task, often leading to errors and disruptions in your data processing. But fear not! Our "Ensure Path" macro is here to simplify your life. Here's why it's a game-changer:

✅ **Dynamic Directory Checking**: This macro intelligently reads incoming directory or full-path file names and dynamically checks the existence of all distinct directories.

✅ **Versatile Path Handling**: Whether you're working with absolute or UNC (Universal Naming Convention) path types, Ensure Path seamlessly accommodates both.

✅ **Comprehensive Status Updates**: Stay in the loop with detailed status reports on all directories, empowering you to monitor your workflow with confidence.

✅ **Swift Error Resolution**: In case of path creation failures, the macro provides clear error messages, enabling quick identification and resolution of issues.

✅ **Workflow Automation**: Designed with a dynamic structure, Ensure Path seamlessly integrates into your workflows, enhancing automation capabilities and eliminating manual path management.

Ready to streamline your data workflow and bid farewell to path-related hassles? Download "Ensure Path" now: [https://bit.ly/CReW_EnsurePath](https://bit.ly/CReW_EnsurePath)

ZIP Output

Effortless File Archiving and Sharing**

If you've ever found yourself drowning in a sea of output files, our "ZIP Output" macro is here to rescue you. This macro simplifies file archiving and sharing:

✅ **Space-Saving Magic**: Combine multiple output files into a single, compressed ZIP folder, reducing clutter and saving valuable disk space.

✅ **Simplified Sharing**: Easily share your results with colleagues, clients, or collaborators by sending a single, compact ZIP file.

✅ **Workflow Efficiency**: Automate the archiving process within your Alteryx workflows, ensuring a smooth and organized data pipeline.

Ready to streamline your file management and sharing? Download "ZIP Output" now: [https://bit.ly/CReW_ZIPOutput](https://bit.ly/CReW_ZIPOutput)

**Get Started Today!**

Empower your data journey with these innovative CReW Macros. You can access and download them directly from the Alteryx Marketplace using the following links:

📁 **Ensure Path**: CReW_EnsurePath

📁 **ZIP Output**: CReW_ZIPOutput

We believe that these macros will greatly enhance your Alteryx experience, making your data workflows more efficient and error-free.

Stay tuned for more updates and additions to our Alteryx Macro collection on ChaosReignsWithin.com. We can't wait to see how these macros help you conquer your data challenges! Happy data crunching! 🚀📊✨

Alteryx Obscura and Wordyx

[ Cross-posting from https://www.linkedin.com/pulse/alteryx-obscura-wordyx-adam-riley-lcmne ]

One of my favourite sessions at the Alteryx Inspire conference in recent years has been the fantastic "Alteryx Obscura" session invented by Alteryx's Steve Ahlgren. For anyone who hasn't seen the session: it is a series of lightening talks from Alteryx ACEs, employees and customers, making Alteryx do fun things that it was never designed to do.

For anyone familiar with my prior work of Chess, Fractals and Conway's Game Of Life in Alteryx, will find it no surprise that Obscura is my kind of session! If you haven't managed to see one of these sessions live at Inspire yet then the Polish User Group did a re-run which you can watch here.

My presentation from 2022 was Wordyx: an implementation of a rather well known word game inside of an Alteryx workflow, played with a text input and browse tool.

The implementation in Alteryx takes a surprisingly few number of tools!

We recently did an internal re-run here at Alteryx and I was asked if I could share the workflow from my Wordyx presentation, which I am very pleased to do here.

https://downloads.chaosreignswithin.com/Wordyx.yxzp

My 2023 presentation Blackjayx was less of a successful workflow (and not one that is yet in a state to share), but I did have the great pleasure of sharing a stage with a "virtual" Sir Ian Livingstone! Certainly a highlight of my career.

Adam Riley

Announcing Alteryx Marketplace | CReW Macros Available via Alteryx!

Alteryx has launched their new Marketplace for Add-Ons.

The Alteryx Marketplace will be the place to find the latest and greatest for your data journey. 

Featured Add-Ons in October include some of the most used tools and macros created and managed by CReW Macros. CReW Macros have already saved Alteryx users hours, if not days, with their efficient and thoughtful tools, and now you can access them on the Alteryx Marketplace as verified and supported Add-Ons. See their power in action in quick demo videos here.

Featured Macros Include:  Ensure Fields, Expect Equal, Expect Records, Expect Zero Records, Grouped Record ID, Control Container Error Check, Only Unique.

More are coming soon!

Ensure Fields                                https://bit.ly/CReW_EnsureFields
Expect Equal                                https://bit.ly/CReW_ExpectEqual
Expect Records                            https://bit.ly/CReW_ExpectRecords
Expect Zero Records                    https://bit.ly/CReW_Expect0Records
Grouped Record ID                      https://bit.ly/CReW_GroupedRecordID
Control Container Error Check    https://bit.ly/CReW_ControlContainerErrorCheck
Only Unique                                 https://bit.ly/CReW_OnlyUnique

Don't worry!  CReW isn't going anywhere.  We are simply making it easier for access to the best macro add-ons directly from Alteryx.  

Cheers,

Mark (and Adam)

Please Subscribe to my youTube channel. 

Control Containers - Part 2 - CReW Container Error Check Macro

In part 1 of this series we looked at the case where we didn't have an input hooked up to our control container in the logging container. This time we are going to look at what we can do with the input anchor connected.

How does the Input Anchor work?

We noted in the last post that without an input anchor the control container works very much like the CReW Runner macro. Well if we connect up the input anchor then the control container works very much like the CReW Conditional Runner macro. That is to say:

• If the Input Anchor is disconnected
• All tools inside run as though they were in a regular container (or straight on the canvas)
• If the Input Anchor is connected
• If the input anchor receives 0 records – The tools in the Control Container DO NOT run
• If the input anchor receives >0 records – The tools in the Control Container run AFTER the last input record is received

Or in pictures

Introducing the Container Error Check Macro

With this we now are very close to replacing the CReW Runner macros with Control Containers! All we need is the S and F outputs provided by the runner macros. We replace this with the Container Error Check Macro.

You can download it from:

http://downloads.chaosreignswithin.com/CReW_ContainerErrorCheck.yxmc

(To install it copy it into the macros folder where you have the CReW macros installed on your machine. For example C:\CReWMacros\Macros)

You connect it directly after a control container and it has the following behaviour:

• If the incoming log messages contain an error all records get pushed out of F (Fail) output
• If incoming log messages have no errors all records get pushed out of S (Success) output

Anyone who has used the Runner macros will find this very familiar.

Conditional Running

Using this macro we can now conditionally run parts of our workflow!

Success case:

Failure case:

This workflow generates a PDF report and then if there are no errors uploads it via an API using the download tool. If on the other hand there are errors, then the file is not uploaded, but an email is sent instead.

Learn More

If you are at Inspire today, please come to my breakout session to learn about this use case and more. 

If you aren't at Inspire, but would like me to present this talk at your local user group them please get in touch and we can set up a date!

Control Containers - Part 1 - Logging Container

Well it is Adam here. I'm back posting on my personal Alteryx Blog after a very long time, but Control Containers are such an important new feature to desktop designer (that I have wanted to see in the product for so long) that I just had to write a blog post about them.

What are Control Containers?

In short Control Containers are the biggest innovation in controlling the order of how things run in an Alteryx workflow since we released the CReW runner macros over 8 years ago. Ever wanted to control what order your outputs write in? Or run one section of a workflow before another? Or how about capturing logging and error messages? Control containers can do all of this and I suspect more things that we havn't even thought of yet. I don't say this lightly, but I think Control Containers are going to be a game changer with what you can do in an Alteryx workflow.

They were released last week as part of the 2023.1 release https://help.alteryx.com/release-notes/designer/designer-20231-release-notes

They are so innovative that myself and Jeff Arnold who worked on the initial design for control containers got a patent granted for the idea https://image-ppubs.uspto.gov/dirsearch-public/print/downloadPdf/20230113187.

This series of blogs is going to teach you about how Control Containers work through a number of example use cases of what you can do with them.

First up the Logging Container!

When Jeff and I were first throwing around ideas for Control Containers, I actually had proposed that the feature would consist of two tools: A Logging Container and a Control Container. The Logging Container being exactly the same as a Control Container, but with no input anchor. In the end we ended up with a single Container with an optional input. So for now let us pretend the Control Container has no input anchor and just focus on what that output anchor does.

First of all like a regular container a Control Container groups together a set of tools. The first difference we will look at though is that new output anchor.

When a workflow runs, all of the log messages for tools inside the container that usually get written to the messages window, also get output as data from the Control Container's outpur anchor. For anyone familiar with the CReW runner macros this use case is very much like the CReW runner macro, but in a single workflow!

So what is this useful for? Well a number of things, but today we will use it to create an audit trail for our workflow. We enclose the entire workflow in a Control Container and then using regular Alteryx tools we add the current datetime and workflow name and write it to a database.

Now every time this workflow runs we can capture this information and build up a historical record of how things change over time!

Next time we will introduce a new CReW macro tool for making this logging data even more useful.

For anyone at Inspire this week in Las Vegas, please come to my session on Wednesday when we will talk about this use case and a lot more!

CReW Expect Records, or "What to expect when you're expecting data"

 This morning I awoke to a WhatsApp message from my friend, @grossal (Alex) asking the following:

[5:12 AM, 8/16/2022] Alex Gross: Is there a CReW "Expect Records" Macros? 

I see a lot of time Use Cases, e.g. Database Connections or files where I need to check if records come in. I think this could also be useful for the SHA256 Macro.

I just had the scenario where I connected to a database and got zero records back - but it didn't crash my workflow because the metadata was still provided by the db.

Alex had already explored the macro updates to either create a cloned copy of the existing macro or to include functionality to choose between expect 0 or expect something.  It would be easy.  

Easy is in the eye of the beholder.  I knew that I'd have to change the image and that change would complicate everything.  I also don't want to interfere with any existing use of the macro.  I opted for another macro.  Of course, I'd opt for Alteryx to include an ERROR condition on tools that they should allow for an ERROR if 0 records are passed through them.

Another vertical workflow!  Instead of using a COUNT RECORDS macro, I opted for using the JOIN tool (configured with join by record position).  When 0 records are present in the I (Input) anchor, then the "Error Message" data is passed to the MESSAGE tool.  That tool is configured as an "ERROR" and passes the contents of the TEXT INPUT tool to the Results panel.

This method is ever so slightly more performant.  But more importantly it doesn't include another macro inside of it.  That simplicity appealed to me.  When I can use fewer tools and less macros (oxymoron) I think that I've got a winner.  After testing with and without AMP and testing just to see that it works I am confident enough to let you, my friends check it out too.

You can find the macro in the Alteryx Community (Gallery).  Right now I'm getting ready for Alteryx & Beer with Alex and the German Alteryx Community.

Cheers,

Mark

P.S. During the Alteryx & Beer session, I was coerced into creating an output anchor for the macro.  If data is present (e.g. NOT an error), the data will pass through the macro.  This was an ask and so it was made to happen.

P.P.S  If you have a LARGE set of data, the "Count Records" method will literally count every record.  This method only requires 1 record to know if things are kosher or not.

CReW SHA256 Macro: Enter the Python

 Did CReW just freeze over?

CReW SHA256 Macro

This wasn't the first rodeo for me, but it is the first time that I've publicly created a macro that uses the python tool in Alteryx.  I pride myself in doing things with BASE-A (100% Alteryx) whenever possible.  That happens virtually 100% of the time.  But until Alteryx adds SHA256 (or other encryption) to their products (note:  a FIPS version of Alteryx is available help.alteryx.com), this macro will get SHA256 encryption on your current (2022.2) version.

Where is the macro?

This link is at the top of the article for those who don't care about the making of the macro....

Alteryx Community Gallery

How does it work?

Select a single data field (String Type) and it will be SHA256 encrypted as:  [SHA256_Output].  There are warnings if EMPTY data is present or if the incoming data is NOT lowercase (the macro will automagically lowercase the input field).  If you don't want these features turned-on, you can disable them in the macro configuration.

The entire record is read and output by the macro.  Only the SHA256Input field is passed into a python tool which uses HASHLIB and encrypts your field.  If the incoming data is Empty(), it is bypassed.  If you have lots of duplicate data, you might want to UNIQUE that data and only pass the unique values through the tool.  It runs about 2,000,000 records per minute on my computer (with AMP on).  Without AMP, it runs about 50% slower for this function.  

Memory and AMP and Python can lead to shortages of memory.  Use caution.  If you experience memory-related issues, you might want to keep things simple.  I ran 200 million records through this process in less than 1.5 hours while testing.

What's inside the box?

Input:

Your incoming data is routed through a common RecordID tool and the entire record circumvents the encryption process and awaits the results in a JOIN tool (append to end of record).

Process-Prepare for Encryption

A formula tool assigns the selected (Interface Drop-Down) string field into the __SHA256Input__ field.  Conversion warnings (MESSAGE) are issued if the input field is Empty().  Conversion warnings are also issued if the input field contains uppercase letters.  The input field is converted to lowercase() for all records.  If these function offend you, you can deselect the execution of the preparation functions in the configuration (via CheckBox that updates the DETOUR).

Python - SHA256 Encryption

Empty data is filtered and then unioned to the results of the encryption.  Data then goes into the Python tool and the following script is executed (please note that the script is saved as a comment inside of a disabled container):

#################################

from ayx import Package

from ayx import Alteryx

import hashlib

#################################

df = Alteryx.read('#1')

#################################

def text_to_sha256(text):

    h=hashlib.new('sha256')

    h.update(text.encode('ascii'))

    return h.hexdigest()

#################################

df['SHA256_Output'] = df.apply(lambda row: text_to_sha256(row['__SHA256Input__']),axis=1)

#################################

Alteryx.write(df,1)

The python tool reads #1 input and outputs via #1.  The imported library, hashlib, is used to created a function that converts text to sha256.  A dataframe for output uses a lambda function to parse the data from __SHA256Input__ and creates your SHA256_Output field.

In an effort to minimize the python use of memory (as tested python uses 50% more memory than Alteryx was consuming), only the RecordID and SHA256Input are passed through this tool.  The tool is configured in "Production Mode".  A copy of the script is saved as a comment as friends of mine tell me is a good practice (sometimes it is lost otherwise).

Error Checking:

What could go wrong?  I can't imagine records being lost in the join, but should it happen there are two (2) CReW Expect 0 macros waiting to throw error messages if any unjoined data tries to escape the macro.

In Closing:

It has been a while since I've posted.  The community gallery is where I've been stashing macros.  Every once in a while I hear questions about the use of CReW while using AMP.  On my to-do list is to look through the parallel-BUD tool and modify it to be AMP compatible.  In reading community posts it seems that it might not work without compatibility mode enabled.  If you have any CReW or AMP feedback/concerns (related to CReW), please let us know.

A special note of thanks to Alex Gross (AYX Community @grossal), ACE and friend who helped to get through some python woes.

Cheers,

Mark