Feedback Forms: Azure Work Items and Github Issues

Published: Feb 28, 2023 by Isaac Johnson

I’ve used Azure DevOps for years now and, over time, have moved most of my workloads to Github. The outstanding part is, and has been, Azure DevOps Work Items. Work Item management in AzDO is just so much richer, in my opinion, than “Issues” in Github.

But this is where I have a wringing of hands and gnashing of teeth - I don’t really go to AzDO very often as my work (these writings) are driven out of Github. Even for my professional employment, I essentially live in Github.

Reviewing the flow

Before we move on to changes, let’s cover the existing flow.

If you click feedback in the upper right of this website

/content/images/2023/02/newfeedback-01.png

You’ll be presented with a simple form that let’s you put in some details

/content/images/2023/02/newfeedback-02.png

This is then processed by some JavaScript that sends it to a public Webhook in AzDO

/content/images/2023/02/newfeedback-03.png

This is picked up by the ‘resources.webhook’ trigger in an Azure Pipeline

/content/images/2023/02/newfeedback-04.png

This will do a few things.

1 Create an issue

The most important part of the flow is that it creates an issue with details the user sent

/content/images/2023/02/newfeedback-04.png

2 Send email(s)

I get an email on any new issues

/content/images/2023/02/newfeedback-06.png

and only if you put a real email address, you’ll get an email as well. You can see the “dontemailme” check here

/content/images/2023/02/newfeedback-07.png

3 Update Slack / Messaging Clients

Lastly, I see a slack message pop up from the builds

/content/images/2023/02/newfeedback-08.png

And Because Microsoft decided to yank back teams for Orgs to roll out a new pay model, I can’t really show you a Teams notification because it’s locked out at the moment

/content/images/2023/02/newfeedback-10.png

I’ll see if I can get through this flow….

/content/images/2023/02/newfeedback-11.png

and then again

/content/images/2023/02/newfeedback-12.png

and then again

/content/images/2023/02/newfeedback-13.png

But I did get back (lots of crashes and restarts - making user to use teh Teams with the blue shield, not white one)

/content/images/2023/02/newfeedback-15.png

4 APM Metrics

Lastly, I capture metrics into Datadog as well

/content/images/2023/02/newfeedback-09.png

Rant about Teams

<RANT>

All right, old man rant time. I’m just about done with Teams…

/content/images/2023/02/newfeedback-21.png

I like the app when used at work. It’s pretty smooth and works on all my devices. But it’s also very very annoying in that they completely refuse to correct the biggest issue that has plagued it form the beginning…

Let’s see if you can spot some differences:

Here is my Discord

/content/images/2023/02/newfeedback-16.png

My Slack

/content/images/2023/02/newfeedback-18.png

and my Teams’s (?Teamsesses)

/content/images/2023/02/newfeedback-17.png

Do you see the horrible annoyance?

The gripe I have is that in the other tools, I can be logged-in to multiple orgs and have one window to deal with but oh no, not with Teams. There i have a personal/O365 app and a “work” or “org” app

/content/images/2023/02/newfeedback-19.png

And the Org one, which appears can now also login in to personal, has to completely log out and log back in to switch organizations

/content/images/2023/02/newfeedback-20.png

This is massively annoying as a professional hobbyist who has ‘work work’ and ‘fun work’, and back in the day as a contractor where i needed to often be in multiple companies’ “Teams” and I did what you saw in the screenshot up top; I quite literally gave up on the fat client and created a browser-per-org setup so i could see all the chats at once.

I’ve spoken with Microsoft people often on this and I always get the same response “just login at the top and switch”. To me this is akin to comparing a cell phone that can have multiple sims to one that has one and are told “just power it off, swap sims and power it on - see, it can do two phone numbers”.

Thanks, Old Man Rant done.

</RANT>

Github Issues

The Github Rest API page for issues is rather clear on how we can use a GH token to engage with issues.

For instance, we can now test the response of “list issues” by using a GH Token (Settings/Developer) to fetch issues. Here I’ll ask for issues in the repo (many created via git-bug):

$ curl -H "Accept: application/vnd.github+json" -H "Authorization: Bearer ghp_asdfsadfasdfasdfasfdasdfasdfasdfasdfasdfasdf" -H "X-GitHub-Api-Version: 2022-11-28" https://api.github.com/repos/idjohnson/jekyll-blog/issues | jq '.[] | .title'
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 45079  100 45079    0     0   108k      0 --:--:-- --:--:-- --:--:--  108k
"Zapper and .internal"
"Post to mastodon"
"Blog - ngrok"
"Hugo - Blogging"
"Blog - Py Func with REST API"
"Blog - Visual Studio Dev Containers"
"Blog - d2, mermaid, other graphing md"
"GCP Cloud Trace"
"Github codespaces"
"Github Artifacts"
"Blog - Visual Studio Dev Containers"
"Blog - d2, mermaid, other graphing md"
"Blog - Py Func with REST API"
"Sumo Logic"
"Cribbl"
"Github Boards (Actions Part 3)"
"Chad P suggests vClusts for OSX/Minikube"

The “problem”, if you want to see it that way, is that the issues are tied to a repo. In my mind, I think more along the lines of JIRA or Azure Work Items in that Issues are related to a “Project” more than a “Repo”.

In Github Parlance, The Project Issues are in an area called “Projects”. Depending on if you are a Github or Github Enterprise user, you may see different options. Generally, i see “Projects” next to “Repositories” in the UI

/content/images/2023/02/newfeedback-22.png

Note: Shout out to my employeer Wellsky who both knows I blog and actively encourages it. I had great support both at Wellsky and Ahead to blog and both companies saw it as a win-win to share knowledge. Just saying - it’s great when you don’t have to avoid work stuff when speaking to the world… That said, be aware, as this is wholly unvetted by anyone but myself, as always, the opinions here are my entirely my own and are not necessarily shared by any current or past employer.

Creating a project

We can create a new project from the Projects page

/content/images/2023/02/newfeedback-23.png

Next, we have to decide between Table style

/content/images/2023/02/newfeedback-24.png

or board

/content/images/2023/02/newfeedback-25.png

After creation, I’ll then edit and give it a proper name, description and basic Readme

/content/images/2023/02/newfeedback-26.png

I can just start typing to create a draft issue

/content/images/2023/02/newfeedback-27.png

But what surprised me, and I think is even way more cool, is that I can pull in Issues from existing Repos.

I’ll click “+” then “Add item from repository”

/content/images/2023/02/newfeedback-28.png

I can pull in just about anything - from Issues to Branches to closed PRs. Here I’ll import an issue (Zappier) and a closed PR (AKV and GCP SM)

/content/images/2023/02/newfeedback-29.png

I was really curious how the PR would look. It added in a comment and even maintained the date

/content/images/2023/02/newfeedback-30.png

We can create a new view and set it to be a board

/content/images/2023/02/newfeedback-31.png

This brings me one of the killer features I like in JIRA which is the ability to view things multiple ways.

In Azure DevOps, I often have a Work Item Query on a Dashboard to accommodate different ways of viewing issues.

/content/images/2023/02/newfeedback-32.png

Using the GitHub CLI

There are a few ways to create Issues in Github. We’ll start by using the Github CLI (gh)

I’ll first install with brew

$ brew install gh

Then Auth in

$ gh auth login
? What account do you want to log into? GitHub.com
? What is your preferred protocol for Git operations? HTTPS
? Authenticate Git with your GitHub credentials? Yes
? How would you like to authenticate GitHub CLI? Login with a web browser

! First copy your one-time code: ASDF-78EA
Press Enter to open github.com in your browser...
[GFX1-]: glxtest: libEGL missing
[2023-02-26T17:59:53Z ERROR glean_core::metrics::ping] Invalid reason code startup for ping background-update
[2023-02-26T18:01:30Z ERROR viaduct::backend::ffi] Missing HTTP status
[2023-02-26T18:01:30Z ERROR viaduct::backend::ffi] Missing HTTP status
[2023-02-26T18:01:30Z ERROR viaduct::backend::ffi] Missing HTTP status
[2023-02-26T18:01:30Z ERROR viaduct::backend::ffi] Missing HTTP status
[2023-02-26T18:01:30Z ERROR viaduct::backend::ffi] Missing HTTP status

✓ Authentication complete.
- gh config set -h github.com git_protocol https
✓ Configured git protocol
✓ Logged in as idjohnson

/content/images/2023/02/newfeedback-33.png

But even with the GH CLI, I can only create issue in the context of repos

builder@DESKTOP-72D2D9T:~/Workspaces/ansible-playbooks$ gh issue create --title "AnotherTest"

Creating issue in idjohnson/ansible-playbooks

? Body <Received>
? What's next? Cancel
Discarding.
builder@DESKTOP-72D2D9T:~/Workspaces/ansible-playbooks$ cd ..
builder@DESKTOP-72D2D9T:~/Workspaces$ gh issue create --title "AnotherTest"
no git remotes found

Creating issues in projects

We now have two ways we can create an issue in a project.

The first is to use the GH CLI

builder@DESKTOP-72D2D9T:~/Workspaces/ansible-playbooks$ gh issue create --title "Feature: AzureDNS Zone" --body "Update
the ClusterIssuer playbook for AzureDNS to pass in a Zone name and RG instead of hardcoding it"

Creating issue in idjohnson/ansible-playbooks

https://github.com/idjohnson/ansible-playbooks/issues/1

which we can see reflected in the GH Issues page for that repository here

/content/images/2023/02/newfeedback-34.png

Or via the REST API

$ curl -X POST -H "Accept: application/vnd.github+json" -H "Authorization: Bearer ghp_asdfasdfasdfasdfasdfasdfasdf" -H "X-GitHub-Api-Version: 2022-11-28" https://api.github.com/repos/idjohnson/ansible-playbooks/issues -d '{"title":"Bug: Some Bug","body":"I'\''m having a problem yo."}'

which i can tie into AKV if I store a token there

$ curl -X POST -H "Accept: application/vnd.github+json" -H "Authorization: Bearer `az keyvault secret show --vault-name idjakv --name GithubToken-MyFull90d -o json | jq -r .value | tr -d '\n'`" -H "X-GitHub-Api-Version: 2022-11-28" https://api.github.com/repos/idjohnson/ansible-playbooks/issues -d '{"title":"Bug: Some Bug","body":"I'\''m having a problem yo."}'

Which, as before, we can see reflected in the issues page for the GH Repo

/content/images/2023/02/newfeedback-35.png

Design decisions

This leads us into two potential paths we can follow for ingestion of external user issues:

  1. A new GH project just to act as a clearing house for new issues. This could be
    1. Public - In which case people could add issues as they saw fit, and see results. However, this also then exposes our full backlog of unaccepted work to the world.
    2. Private - We still follow the form to ingest work, then clear it from a Private repo into either specific repos where the work will be done -OR- if repo independent, process it in the ingestion repo.
  2. Use the single Primary (Jekyll-blog) private repo to ingest work.
    1. New issues get prefixed as “external” for tracking
    2. This has the advantage of putting all work into a single bucket to then process accordingly.

If we follow a path with private repositories, we likely need some kind of release notes or releases page to show what work was handled and when. The Public repo path has the advantage of letting people (who are willing to auth into GH) to put in comments and have a conversation. We can still add “anonymous” entries, they will just come in as our user (idjohnson in my case).

One other topic we need to cover is whether we pull the plug on adding more work into the Azure Pipelines. There is nothing wrong, mind you, but we could move the work into a Github action instead.

/content/images/2023/02/Notes_230225_131858.jpg

I need to add the Azure DevOps SP to access the AKV

/content/images/2023/02/newfeedback-36.png

Now I can pick my AKV in the Azure Pipelines Library and select a secret with my GH Token

/content/images/2023/02/newfeedback-37.png

I then added the Group Var to the pipeline:

variables:
- group: AZDOAutomations
- group: AzureKeyVaultGH

Then I added a step to use it

- script: |
    cat >rawDescription <<EOOOOL
    $
    EOOOOL

    cat >rawSummary <<EOOOOT
    $
    EOOOOT

    sed -i "s/'/'\\\''/g" rawDescription
    sed -i 's/"/\\"/g' rawDescription
    sed -i "s/'/'\\\''/g" rawSummary
    sed -i 's/"/\\"/g' rawSummary

    cat >wiDetails.json <<WIDEETS
    {
      "title":"`cat rawSummary`",
      "body":"`cat rawDescription`"
    }
    WIDEETS

    cat wiDetails.json

    curl -X POST -H "Accept: application/vnd.github+json" \
         -H "Authorization: Bearer $(GithubToken-MyFull90d)" \
         -H "X-GitHub-Api-Version: 2022-11-28" \
         https://api.github.com/repos/idjohnson/jekyll-blog/issues -d @wiDetails.json
    
  displayName: Create GH Ticket

The first time through, I need to approve it

/content/images/2023/02/newfeedback-38.png

I’ll try a test ticket

/content/images/2023/02/newfeedback-39.png

This kicked off a build

/content/images/2023/02/newfeedback-40.png

Which did succeed

/content/images/2023/02/newfeedback-41.png

Funny thing is, while testing, I had a few typos and after several failed builds, I trigger a PagerDuty call on myself!

/content/images/2023/02/newfeedback-42.png

I can see the Work Item in AzDO created

/content/images/2023/02/newfeedback-43.png

and now also an Issue in Github

/content/images/2023/02/newfeedback-44.png

However, more complicated suggestions tend to fall down

/content/images/2023/02/newfeedback-45.png

Little Bobby Tables…

/content/images/2023/02/azdoghgcpakv-45.png

In working out the proper way to sanitize my input, I kept triggering Pagerduty.

I’m actually happy that I alert in such a severe way when my input forms failed, but it gave me a real start seeing PD Outgoing Numbers phone calls late at night when working on this (thinking it was an employer outage).

I actually put in a Maintenance Window in Pagerduty to avoid alerts while I tweaked syntax

/content/images/2023/02/azdoghgcpakv-44.png

Sanitized Input

I arrived at the following block to sanitize the input

   data="$( jq -nc --arg title "$" \
      --arg body "$" '$ARGS.named')"
   cat >emailJson2.json <<EOTT
   $data
   EOTT

For the Azure Work Items, this looked like


- script: |
    set -x
    echo Add other tasks to build, test, and deploy your project.
    echo "userId: $"
    echo "summary: $"
    echo "description: $"

    data="$( jq -nc --arg title "$" --arg body "$" '$ARGS.named')"
    cat >emailJson2.json <<EOTT
    $data
    EOTT

    cat >$(Pipeline.Workspace)/createwi.sh <<EOL
    set -x
    export AZURE_DEVOPS_EXT_PAT=$(AZDOTOKEN)
    az boards work-item create --title `echo $data | jq .title | tr -d '\n'` --type Feature --org https://dev.azure.com/princessking --project ghost-blog --discussion 'requested by $' --fields System.Description=`echo $data | jq .body | tr -d '\n'` > azresp.json
    EOL
    chmod u+x createwi.sh

    echo "createwi.sh:"
    cat $(Pipeline.Workspace)/createwi.sh

    echo "have a nice day."
  displayName: 'Check webhook payload'

- task: AzureCLI@2
  displayName: 'Create feature ticket'
  inputs:
    azureSubscription: 'My-Azure-SubNew'
    scriptType: 'bash'
    scriptLocation: 'scriptPath'
    scriptPath: '$(Pipeline.Workspace)/createwi.sh'
  condition: always()

and the GH Actions as

- script: |
    
    data="$( jq -nc --arg title "$" --arg body "$" '$ARGS.named')"
    cat >emailJson2.json <<EOTT
    $data
    EOTT

    curl -X POST -H "Accept: application/vnd.github+json" \
         -H "Authorization: Bearer $(GithubToken-MyFull90d)" \
         -H "X-GitHub-Api-Version: 2022-11-28" \
         https://api.github.com/repos/idjohnson/jekyll-blog/issues -d @emailJson2.json
    
  displayName: Create GH Ticket
  condition: always()

That meant submitting the more complicated form of:

/content/images/2023/02/newfeedback-45.png

generated the Work Item:

/content/images/2023/02/azdoghgcpakv-46.png

And GH Issue

/content/images/2023/02/azdoghgcpakv-47.png

And since the user did put in a request to be emailed, they would get an alert:

/content/images/2023/02/azdoghgcpakv-48.png

I wanted one last little change - add the requestor to the GH Issue and fix the newlines in the Azure Work Item

The final pipeline looked like:

resources:
  webhooks:
    - webhook: feedbackForm
      connection: feedbackForm
      
pool:
  vmImage: ubuntu-latest


variables:
- group: AZDOAutomations
- group: AzureKeyVaultGH

steps:
- script: |
    set -x
    echo Add other tasks to build, test, and deploy your project.
    echo "userId: $"
    echo "summary: $"
    echo "description: $"

    cat >rawDescription <<EOOOOL
    $
    EOOOOL

    cat >rawSummary <<EOOOOT
    $
    EOOOOT

    cat rawDescription | sed ':a;N;$!ba;s/\n/<br\/>/g' > inputDescription2
    cat rawSummary | sed ':a;N;$!ba;s/\n/ /g' > inputSummary2
    
    echo "input summary2: `cat inputSummary2`"
    echo "input description2: `cat inputDescription2`"

    data="$( jq -nc --arg title "`cat rawSummary`" --arg body "`cat inputDescription2`" '$ARGS.named')"

    cat >$(Pipeline.Workspace)/createwi.sh <<EOL
    set -x
    export AZURE_DEVOPS_EXT_PAT=$(AZDOTOKEN)
    az boards work-item create --title `echo $data | jq .title | tr -d '\n'` --type Feature --org https://dev.azure.com/princessking --project ghost-blog --discussion 'requested by $' --fields System.Description=`echo $data | jq .body | tr -d '\n'` > azresp.json
    EOL
    chmod u+x createwi.sh

    echo "createwi.sh:"
    cat $(Pipeline.Workspace)/createwi.sh

    echo "have a nice day."
  displayName: 'Check webhook payload'

- task: AzureCLI@2
  displayName: 'Create feature ticket'
  inputs:
    azureSubscription: 'My-Azure-SubNew'
    scriptType: 'bash'
    scriptLocation: 'scriptPath'
    scriptPath: '$(Pipeline.Workspace)/createwi.sh'
  condition: always()

- script: |
    
    data="$( jq -nc --arg title "$" --arg body "$ :: Requested by $" '$ARGS.named')"
    cat >emailJson2.json <<EOTT
    $data
    EOTT

    curl -X POST -H "Accept: application/vnd.github+json" \
         -H "Authorization: Bearer $(GithubToken-MyFull90d)" \
         -H "X-GitHub-Api-Version: 2022-11-28" \
         https://api.github.com/repos/idjohnson/jekyll-blog/issues -d @emailJson2.json
    
  displayName: Create GH Ticket
  condition: always()

- script: |
    sudo apt-get update
    sudo apt-get install -y s-nail || true
    
    set -x
    export WIID=`cat azresp.json | jq -r '.id' | tr -d '\n'`
    cat rawSummary |sed ':a;N;$!ba;s/\n/ /g' | sed 's/"/\\"/g' > emailSummary
    
    export USERTLD=`echo "$" | sed 's/^.*@//'`

    if [[ "$USERTLD" == "dontemailme.com" ]]; then
       # do not CC user
       echo "<h1>New Feature Requested</h1><p>user $ has requested "`cat emailSummary`"</p><p>https://dev.azure.com/princessking/ghost-blog/_workitems/edit/$WIID/</p><br/><br/>Kind Regards,<br/>Isaac Johnson" | s-nail -s "Blog: Feature $WIID Requested" -M "text/html" -S smtp=email-smtp.us-east-1.amazonaws.com:587 -S smtp-use-starttls -S ssl-verify=ignore -S smtp-auth=login -S smtp-auth-user=$(SMTPAUTHUSER) -S smtp-auth-password=$(SMPTAUTHPASS) -c isaac.johnson@gmail.com -r isaac@freshbrewed.science isaac.johnson@gmail.com
    else
       # may not work
       echo "<h1>New Feature Requested</h1><p>user $ has requested "`cat emailSummary`"</p><p>https://dev.azure.com/princessking/ghost-blog/_workitems/edit/$WIID/</p><br/><br/>Kind Regards,<br/>Isaac Johnson" | s-nail -s "Blog: Feature $WIID Requested" -M "text/html" -S smtp=email-smtp.us-east-1.amazonaws.com:587 -S smtp-use-starttls -S ssl-verify=ignore -S smtp-auth=login -S smtp-auth-user=$(SMTPAUTHUSER) -S smtp-auth-password=$(SMPTAUTHPASS) -c $ -r isaac@freshbrewed.science isaac.johnson@gmail.com
    fi
    
    cat >valuesLA.json <<EOLA
    {
        "requestorEmail": "$",
        "emailSummary": "`cat emailSummary`",
        "workitemID": "$WIID",
        "workitemURL": "https://dev.azure.com/princessking/ghost-blog/_workitems/edit/$WIID/",
        "desitinationEmail": "isaac.johnson@gmail.com"
    }
    EOLA

    #curl -v -X POST -d @valuesLA.json -H "Content-type: application/json" "https://prod-34.eastus.logic.azure.com:443/workflows/asdfasdfasdfasdfsadfasdfasdf/triggers/manual/paths/invoke?api-version=2016-10-01&sp=%2Ftriggers%2Fmanual%2Frun&sv=1.0&sig=asdfasdfasdfasdfasdfasdfasfasdf"
    
  displayName: 'Email Me'

Then tested with

/content/images/2023/02/azdoghgcpakv-49.png

which fixed the GH issue

/content/images/2023/02/azdoghgcpakv-51.png

And the Azure DevOps Work Item

/content/images/2023/02/azdoghgcpakv-52.png

And emails

/content/images/2023/02/azdoghgcpakv-56.png

I didn’t really cover it in detail, but you can see at one point I used an Azure Logic app, which is nice, but rather pricey. I use instead s-nail to trigger SES emails from AWS

This is significantly cheaper

/content/images/2023/02/azdoghgcpakv-57.png

Also, the feedback form itself, which is mostly meant to pass a JSON payload is covered in the first writeup of this from a couple years ago which uses this GH project to render out a static form.

Cleanup

Since I debugged with Live Projects/Repos, I have some cleanup to do

/content/images/2023/02/azdoghgcpakv-53.png

and in Github

/content/images/2023/02/azdoghgcpakv-54.png

I see Datadog is now happy again with the Pipelines

/content/images/2023/02/azdoghgcpakv-55.png

Summary

This really covered two improvements to the original article; Updating the pipelines to sanitize the input - complicated feedback was likely dropped in the past, unfortunately. The second was to add Github Issues to the mix.

The part noted in the left of the hand drawn diagram was my next to-do - a completion flow. It would be nice, for those that leave feedback, to be notified on updates (that thing you asked for is now posted). I think doing that manually, for now, is fine as I really do not get much feedback (save for my buddy pinging me for Pho which is always a great idea).

Azure DevOps AzDO Feedback Github Issues Azure Work Items

Have something to add? Feedback? Try our new forums

Isaac Johnson

Isaac Johnson

Cloud Solutions Architect

Isaac is a CSA and DevOps engineer who focuses on cloud migrations and devops processes. He also is a dad to three wonderful daughters (hence the references to Princess King sprinkled throughout the blog).

Theme built by C.S. Rhymes