
Microsoft Authentication Library or MSAL is a library that enables web applications to authenticate users against Azure Active Directory (AAD). This setup can be crucial for applications requiring secure identity management and access control, especially within organizations using Microsoft services.

MSAL JS library supports protocols such as OAuth2 and OpenID Connect, making it easier to implement for web applications.

So let's begin.

### Registering Your Application with Azure AD

First, you need to register your application with Azure AD to obtain valid credentials for authentication.

1. Sign into the Azure Portal: Navigate to Azure Portal.
2. Create a New App Registration: Go to Azure Active Directory > App registrations > New registration. Make sure you select SPA applications.
3. Configure Redirect URIs: For development, and in my case, I used `http://localhost:3000/`. Ensure not to enable Implicit Grant or Hybrid Flows.

![Azure-SPA-Registered](/images/azure-spa-register.jpeg)

### Setting Up Your Next.js Project

Initialize a new Next.js project if you haven't already:

```bash
npx create-next-app msal-next-auth --use-npm
cd msal-next-auth
npm install @azure/msal-react @azure/msal-browser
```

### Configuring MSAL in Your Next.js App

Create a configuration file for MSAL and initialize the MSAL instance.

In my case I created a file named `msal.js` in `services/` directory:

```javascript
import { PublicClientApplication } from "@azure/msal-browser";

const msalConfig = {
    auth: {
        clientId: process.env.NEXT_PUBLIC_AZURE_AD_CLIENT_ID,
        authority: `https://login.microsoftonline.com/${process.env.NEXT_PUBLIC_AZURE_AD_TENANT_ID}`,
        redirectUri: process.env.NEXT_PUBLIC_REDIRECT_URI
    }
};

const msalInstance = new msal.PublicClientApplication(msalConfig);
export { msalInstance };
```

Ensure you have set the environment variables in `.env.local`:

```bash
NEXT_PUBLIC_AZURE_AD_CLIENT_ID=your-client-id
NEXT_PUBLIC_AZURE_AD_TENANT_ID=your-tenant-id
NEXT_PUBLIC_REDIRECT_URI=/
```

Note that the `redirectUri` should match the one you configured in Azure AD.
Also, note that NEXT_PUBLIC env vars (browser env vars) are being applied only in during build time and not in runtime. Which means if these env vars are environment dependent, you should take it into account in your CI that handles the application build and deployment.

Next, Modify `_app.js` to Use MSAL Provider:

```javascript
import { MsalProvider } from '@azure/msal-react';
import { msalInstance } from '@/services/msal';
import '.@/styles/globals.css';

function MyApp({ Component, pageProps }) {
  return (
    <MsalProvider instance={msalInstance}>
      <Component {...pageProps} />
    </MsalProvider>
  );
}

export default MyApp;
```

### Adding Authentication to Your Pages

Here you can use MSAL React hooks which makes your life easier and add authentication functionality.

For example, in `pages/index.js`:

```javascript
import { AuthenticatedTemplate, UnauthenticatedTemplate, useMsal } from '@azure/msal-react';
import React from 'react';

function SignInButton() {
    const { instance } = useMsal();
    return <button onClick={() => instance.loginRedirect()}>Sign In</button>;
}

function WelcomeUser() {
    const { accounts } = useMsal();
    const username = accounts[0]?.name || 'Guest';
    return <p>Welcome, {username}</p>;
}

export default function Home() {
    return (
        <div>
            <AuthenticatedTemplate>
                <WelcomeUser />
            </AuthenticatedTemplate>
            <UnauthenticatedTemplate>
                <SignInButton />
            </UnauthenticatedTemplate>
        </div>
    );
}
```
Note that you can use the token acquired from MSAL to call your API with authentication header (if required) or use any other attribute from the token object such as name from the example.

You can also use the `useIsAuthenticated` hook to check if the user is authenticated and show different content accordingly. 

```javascript
import { useIsAuthenticated } from '@azure/msal-react';

export default function Home() {
    const isAuthenticated = useIsAuthenticated();
    return (
        <div>
            {isAuthenticated ? <WelcomeUser /> : <SignInButton />}
        </div>
    );
}
```

In one of my projects at work I created a custom hook to use the token in the API calls:

```javascript
import { useMsal } from '@azure/msal-react';

export const useAccessToken = () => {
  const { instance, accounts } = useMsal();
  const account = accounts[0];

  const getAccessToken = async () => {
    try {
      const request = {
        scopes: [`${process.env.NEXT_PUBLIC_CLIENT_ID}/.default`],
        account: account,
        redirectUri: '/',
      };

      const response = await instance.acquireTokenSilent(request);
      return response.accessToken;
    } catch (error) {
        console.log('error', error)
      if (error instanceof InteractionRequiredAuthError) {
        instance.acquireTokenRedirect(request);
      } else {
        throw error;
      }
    }
  };

  return getAccessToken;
};
```

And just for the sake of the example, one of components which uses the the custom hook which fetches the accessToken from AAD and sending as Authentication header in one if its API calls:

```javascript
import React, { useState, useEffect } from 'react';
import { useMsal } from '@azure/msal-react';

import { useAccessToken } from '@hooks/useToken';

const DeviceTable = () => {
  const getAccessToken = useAccessToken();
  const [regions, setRegions] = useState({});
  const [error, setError] = useState(null);

  useEffect(() => {
    let apiUrl = '';

    const fetchConfigAndData = async () => {
      try {
        const accessToken = await getAccessToken();

        const regionsData = await fetch(`${apiUrl}/api/v1/devices`, {
          headers: {
            'Authorization': `Bearer ${accessToken}`
          }
        }).then(res => res.json());
        setRegions(regionsData);
      } catch (error) {
        setError(error);
      }
    };

    fetchConfigAndData();
  }, []);
```

Keep in mind that i'm not an experienced frontend developer, so the code above might not be the best practice, but it works for me. I'll be happy to hear your feedback and suggestions. I'm mostly available on [Linkedin](https://www.linkedin.com/in/kazaz-or/).


Learn how to integrate Azure AD authentication into your Next.js app using MSAL library for security and user management.

Integrating Azure AD Authentication in Your Next.js App Using MSAL



The common purposes of scheduling Lambda functions is to automate tasks such as nightly data backups, data fetching and processing, or any routine task that needs to run on a scheduled basis.

If you're using AWS as a cloud provider, once of the most common services to schedule your lambdas is EventBridge (formerly known as CloudWatch Events).

To show and example of how to create such thing, we'll use Terraform.

First, lets create the rule itself (eventually it'll be registered in EventBridge):

```hcl
resource "aws_cloudwatch_event_rule" "schedule" {
  name                = "${var.name_prefix}-schedule"
  description         = "Schedule for Updater Lambda Function (ap-vpn-console)"
  schedule_expression = var.schedule_expression
}
```
__name:__ A unique identifier for the rule, using a variable for flexibility.

__description:__ Clear description of what the rule does.

__schedule_expression:__ When the Lambda will be triggered. Use the `schedule_expression` variable to easily change the timing without modifying the code directly.

Just in case you're not familiar with cron expressions, here is an example:

```hcl
variable "schedule_expression" {
  description = "The schedule expression for the lambda function"
  type        = string
  default     = "cron(0 2,10 * * ? *)"
}
```
This cron expression means the Lambda will run at 2:00 AM and 10:00 AM (UTC) every day.

![schedule-rule](/images/eventbridge-lambda-rule.jpeg)


Now let's hook it with our Lambda function:

```hcl
resource "aws_cloudwatch_event_target" "schedule_lambda" {
  rule      = aws_cloudwatch_event_rule.schedule.name
  target_id = "processing_lambda"
  arn       = aws_lambda_function.lambda.arn
}
```

__rule:__ Links the target to the previously defined rule.

__target_id:__ An identifier for the target.

__arn:__ Amazon Resource Name for the Lambda function to execute.


Last thing we might want to do, is to allow EventBridge to invoke the Lambda function by defining its permissions:

```hcl
resource "aws_lambda_permission" "allow_events_bridge_to_run_lambda" {
  statement_id  = "AllowExecutionFromCloudWatch"
  action        = "lambda:InvokeFunction"
  function_name = aws_lambda_function.lambda.function_name
  principal     = "events.amazonaws.com"
}
```

__statement_id:__ A unique identifier for the statement.

__action:__ Specifies the allowed action.

__function_name:__ Links to the Lambda function.

__principal:__ The AWS service that is allowed to perform the action.


Schedule your AWS Lambda tasks using Terraform and AWS EventBridge to automate routine tasks and improve efficiency.

Scheduling Lambdas Using AWS EventBridge and Terraform


Every web developer, either frontend or backend, may have come across this CORS policy violation (Cross-Origin Resource Sharing) error message at least once in their career.

The error itself is actually pretty informative. It basically tells you that the client-side is not one of the "whitelisted" origins to access the data being fetched. This is a security mechanism that browsers implement to prevent malicious users from accessing the data from a different origin.

Let's dive in a little bit.

First, what is CORS exactly? CORS stands for Cross-Origin Resource Sharing, which is a security mechanism that allows/prevents one origin to access a resource from a different origin. This is something that the server has control over to restrict who has access to the resource, how they can access the data, for example, which HTTP methods, HTTP headers, etc are allowed.
At its core, you can treat it as a protocol allowing web applications to request resources from different domains securely. It's a dance of headers and requests, where the server dictates who can access its resources.


__Same Origin Policy vs Cross Origin Resource Sharing__

The Same-Origin Policy (SOP) is a critical security mechanism that restricts web pages from making requests to a different origin than the one that served the web page. CORS, on the other hand, is a tool to relax those restrictions under controlled conditions. It allows servers to specify who can access their assets and how.

__CORS Mechanism__

CORS works by adding new HTTP headers that allow servers to declare which origins are permitted to access the resources. For example, the `Access-Control-Allow-Origin` header can specify an allowed origin, or it can use `*` to allow requests from any origin. However, using `*` in most cases is discouraged for sensitive resources due to security concerns.



Let's take a look at an example using NodeJS & Express:

```javascript
const express = require('express');
const cors = require('cors');
const app = express();

app.use(cors());

// CORS with specific options
const corsOptions = {
  origin: 'https://example.com',
};

app.get('/data', cors(corsOptions), (req, res) => {
  res.json({ message: 'This route is CORS-enabled for an allowed origin.' });
});

app.listen(3000, () => console.log('Server running on port 3000'));
```

Here we're basically using the `cors` middleware that is applied globally to allow Cross-Origin Resource Sharing (CORS) from any origin, enabling client applications from different domains to access the server. Additionally, it showcases a more complex setup where CORS settings are customized for a specific route, allowing requests only from a specified origin and setting specific methods and headers.

Now, let's take a look at another example, but this time using Python & FastAPI:

```python
from fastapi import FastAPI
from fastapi.middleware.cors import CORSMiddleware

app = FastAPI()

# Simple CORS
app.add_middleware(
    CORSMiddleware,
    allow_origins=["*"],
    allow_credentials=True,
    allow_methods=["*"],
    allow_headers=["*"],
)

# CORS for specific origins
origins = [
    "http://example.com",
]
app.add_middleware(
    CORSMiddleware,
    allow_origins=origins,
    allow_credentials=True,
    allow_methods=["GET"],
    allow_headers=["X-Custom-Header"],
)

@app.get("/data")
async def get_data():
    return {"message": "This route is CORS-enabled for specific origins."}
```

This simple snippet demonstrates two approaches: a global application of CORS allowing all origins, methods, and headers, and a more restrictive approach where CORS is configured for specific origins, methods, and headers on all routes, illustrating how to control access to your API more granularly.

Implementing CORS correctly is a balancing act. Developers must ensure their web applications are accessible to legitimate users and services while protecting against unauthorized access. Understanding CORS is crucial for any developer working on the web, as it directly impacts the security and functionality of web applications.


I can assure you that one of the first issues you'll encounter when testing your newly developed product will be CORS. So what is CORS exactly?

CORS - Introduction For The New FE/BE Developer


Ensuring consistent application performance across multiple platforms is a cornerstone of quality software delivery. In this post, I'll share my approach for using GitHub Actions to automate testing across a variety of platforms, drawing from a real use case at my workplace.

Our solution utilizes GitHub Actions, a CI/CD service that enables automation of build, test, and deployment workflows directly within GitHub. The highlight of the approach is the dynamic matrix strategy, which handles both scheduled runs and on-demand tests across specified platforms.

__Scheduled and On-demand Testing__

In this example, the workflow is being triggered automatically at 22:00 every day for nightly tests. Additionally, we enabled manual triggers for specific platforms, allowing for targeted testing as needed. This flexibility is crucial for maintaining a fast-paced development cycle without compromising on quality.

```yaml
on:
  schedule:
    - cron: '0 22 * * *'
  workflow_dispatch:
    inputs:
      platform:
        description: 'Choose platform to run the test on'
        required: true
        type: choice
        options:
          - webos
          - tizen
          - playstation4
          - playstation5
          - xbox
```

This configuration leverages the `schedule` and `workflow_dispatch` events in GitHub Actions. The `cron` syntax schedules automated test runs, while `workflow_dispatch` allows for manual execution with platform-specific parameters.

__Dynamic Test Matrix Configuration__

The core of our workflow dynamically adjusts the test matrix based on the context of the trigger—whether it's a scheduled job covering all platforms or a manual job focusing on a specific platform.

```yaml
jobs:
  setup-test:
    runs-on: ubuntu-latest
    outputs:
      platform: ${{ steps.set-matrix.outputs.platform }}
    steps:
      - id: set-matrix
        run: |
          if [ "${{ github.event_name }}" = "schedule" ]; then
            echo 'platform=["webos", "tizen", "playstation4", "playstation5", "xbox"]' >> $GITHUB_ENV
          else
            echo 'platform=["${{ github.event.inputs.platform }}"]' >> $GITHUB_ENV
```

In this snippet, we employ conditional logic to populate the platform variable differently based on the trigger type. For scheduled runs, we test across all platforms, while manual triggers allow for focused testing on a selected platform. This strategy ensures that resources are allocated efficiently, focusing our testing efforts where they are most needed.
Of course, you could always do that with an external script or with a composite action, but this is a simple and effective way to achieve the same result.

__Executing the Tests with Dynamic Configuration__

Our approach to executing tests in this workflow benefits significantly from the dynamic nature of GitHub Actions, especially when it comes to handling multiple platforms which can be quite different from each other. This flexibility is further extended into our application's `package.json` file, where we define specific scripts for our testing needs. The `package.json` plays a crucial role in integrating the GitHub environment variables directly into our test execution process.

Without getting into the tests code, you can imagine that each platform has its own specific tests and configurations, and that's why we need to run the tests separately for each platform. This is where the dynamic matrix comes into play.


```yaml
- name: Run tests
  working-directory: ./src/lr-launcher-e2e
  env:
    npm_config_env: ${{ matrix.platform }}
  run: npm test
```

Here, the `npm test` command is executed within the context of each platform specified by the dynamic matrix. This ensures that our E2E tests are relevant and efficient, automatically adapting to the complexities of multi-platform support.

Here's a look at the relevant part of our `package.json` file:

```json
{
  "name": "lr-launcher",
  "version": "1.0.0",
  "scripts": {
    "pre-test": "node ./utils/setupTest.js --",
    "test": "mocha \"tests/*.test.js\" \"tests/$npm_config_env/*.test.js\" --no-timeouts --retries 1 --exit"
  },
  "dependencies": {
    "axios": "^1.6.2",
    "mocha": "^10.2.0",
  }
}
```
This script utilizes the `npm_config_env` environment variable, which we set dynamically based on the selected platform in our GitHub Actions workflow. By incorporating `$npm_config_env` into the mocha command, we ensure that the test runner picks up not only the general tests from tests/*.test.js but also platform-specific tests located in directories named after each platform (e.g., tests/webos/*.test.js).

This setup allows us to maintain a clear and organized test structure where platform-agnostic tests are separated from those that are platform-specific. It ensures that when our GitHub Actions workflow runs, it only executes the relevant tests for the specified platform, making our testing process both efficient and comprehensive.


Automate cross-platform testing with GitHub Actions, using dynamic matrices for efficient, targeted tests.

GitHub Actions for Dynamic Cross-Platform Testing


Makefiles are powerful tools in a developer's arsenal, capable of significantly improving efficiency and consistency in the build process. By understanding and utilizing Makefiles for Docker and Terraform, developers can streamline their workflows, reduce repetitive tasks, and focus more on the creative aspects of software development.

In this article, we'll explore the basics of Makefiles and how they can be used to automate the build process with examples of Docker and Terraform commands usage.

### What is a Makefile?

Incorporating a Makefile into your project can greatly streamline various development tasks, such as managing dependencies, running tests, building documentation,formatting code, performing static analysis, build/compile/transpile, cleaning up temporary files and more.

The flexibility of Makefiles allows you to automate routine processes in a structured and efficient manner. Here's a more detailed look at how you can leverage Makefiles in your projects:


### Basic Structure of a Makefile

A Makefile typically consists of targets, prerequisites, and recipes. Targets represent the goals you want to achieve, which could be actions like running your app or installing dependencies. Prerequisites are files or other targets needed before a target can be executed. Recipes are shell commands that make will execute.

For instance, you might have a `run` target to execute your Python application, a `setup` target for installing dependencies listed in `requirements.txt`, and a `clean` target for removing Python bytecode files or the `__pycache__` directory.

### Advanced Techniques

__Default Goals and Help Messages:__ Setting a default goal, like `.DEFAULT_GOAL := help`, allows you to display a helpful message when make is run without a target.

__Virtual Environments:__ Integrating virtual environments in your Makefile ensures that the right Python environment and dependencies are used. This can be crucial in projects with specific version requirements or multiple dependencies.

__Injecting Paths into `PYTHONPATH:`__ This advanced trick allows Python to search for packages in specified file system paths without needing to install them, which can be particularly useful in projects with heavy dependencies or multiple branches.


### Some Practical Examples

__Running Tests with Coverage:__ You can create a coverage target to run tests with coverage, specifying the commands for running the tests and generating a coverage report.

__Linting and Static Analysis:__ A lint target can be set up to run various linting and static analysis tools like flake8, pylint, and mypy on your code.

__Task Dependencies and Parameters:__ Makefiles allow you to specify dependencies between tasks. For example, a test target might depend on a lint target. You can also pass parameters to tasks, like specifying an IP and port for a serve task.

__Makefile for Docker Commands:__ 

By leveraging Makefiles in your Python projects, you not only save time but also create a more robust and maintainable development environment. These examples and advanced features illustrate just how versatile and powerful Makefiles can be in automating and streamlining your Python development workflow.

```makefile
export TOP := $(shell git rev-parse --show-toplevel)
export IMAGE_NAME := my-docker-image
export CONTAINER_NAME := my-container-image

build:
	docker build --build-arg JFROG_USERNAME=${JFROG_USERNAME} --build-arg JFROG_ARTIFACTORY_TOKEN=${JFROG_ARTIFACTORY_TOKEN} -t $(IMAGE_NAME) .

run:
	$(TOP)/scripts/run_docker.sh &

exec:
	docker exec -it $(CONTAINER_NAME) /bin/sh

delete:
	docker stop $(CONTAINER_NAME)
	docker rm $(CONTAINER_NAME)
```
In this example the `build` command creates a Docker image with build arguments (`make build`).

The `run` command runs a script to start the container in the background (`make run`). It runs a bash script that starts the container in the background and saves the container ID in a file.

The `exec` provides an interactive shell within the container for debugging proposes (`make exec`).

The `delete` stops and removes the container (`make delete`).

__Makefile for Terraform Commands:__

Terraform, an infrastructure as code software tool, also benefits from Makefiles, its commands can become quite complex. Here's an example on why it is useful to use Makefiles for Terraform:

```makefile
account := dev
namespace ?= $(shell echo $(USER) | awk '{print tolower($0)}')
service ?= $(shell basename `git rev-parse --show-toplevel`)

ifeq ($(account), dev)
    AWS_PROFILE := 123456
else ifeq ($(account), stage)
    AWS_PROFILE := 654321
else
    $(error Invalid account specified: $(account))
endif

init:
	terraform init -reconfigure \
		-backend-config=config/$(account)/backend.tfvars -backend-config=key=services/$(namespace)/$(service).tfstate

plan:
	terraform plan -out /tmp/$(service).tfplan \
		-var-file=config/$(account)/terraform.tfvars -var-file=config/$(account)/service.tfvars \
		-var "namespace=$(namespace)" -var "service_name=$(service)"

apply:
	terraform apply /tmp/$(service).tfplan

validate:
	@terraform validate

destroy:
	terraform destroy \
		-var-file=config/$(account)/terraform.tfvars -var-file=config/$(account)/service.tfvars \
		-var "namespace=$(namespace)" -var "service_name=$(service)"
```

This Makefile automates various Terraform commands like `init` which initializes a new or existing Terraform configuration (`make init`).
This command is particularly used to initialize the working directory containing Terraform configuration files, with -reconfigure to force re-initialization. It specifies backend configuration variables and the state file location.

The `plan` command creates an execution plan. This command creates a Terraform plan and outputs it to a temporary file. It specifies variable files and sets additional variables like namespace and service_name (`make plan`).

The `apply` command applies the changes required to reach the desired state of the configuration (`make apply`).

The `validate` command is used to validate the syntax and configuration of the Terraform files (`make validate`).

The `destroy` command is used to destroy all resources managed by the current Terraform configuration (`make destroy`).

In conclusion, Makefiles stand as a fundamental and powerful tool in software development, offering a versatile approach to automating and streamlining various aspects of the build process. Whether you're working with Docker containers or managing infrastructure with Terraform, Makefiles bring a level of efficiency and consistency that can significantly enhance your workflow. By effectively leveraging Makefiles, developers not only save valuable time but also ensure a more robust and maintainable development process, making them an essential component in any modern software project.


Makefiles are a cornerstone tool in the software development process, particularly useful for streamlining and automating the build process.

Why Makefile is One of My Essentials Tools in Every Software Project


At work, I manage a repository for a front-end application, which includes Markdown files that are compiled into a documentation portal.

Documentation, especially in Markdown format, is more than just words on a page. In my case, it's a user's guide, a developer's reference, and sometimes, even a product's first impression. Therefore, ensuring its quality, consistency, and accuracy is critical, and this is where Markdown linters come into play.

### Essential Tools for Markdown Linting I Used

Our approach involves three key tools, each serving a unique purpose:

__Markdownlint:__ For stylistic consistency and syntax correctness.

__Markdown-link-check:__ To validate all hyperlinks are functional.

__cspell__ (Code Spell Checker): For ensuring correct spelling throughout your documentation.

In my case, we used to run these linters on each pull request/commit that was uploaded to the repository.

### Linters

__Markdownlint:__

```bash
npm install markdownlint-cli
```

Ensures Markdown files follow best practices and maintain a consistent style. Of course, as anything the following different rules, it can be customized to your needs.

Markdown lint follows its defined rules, you can find it [here](https://github.com/markdownlint/markdownlint/blob/main/docs/RULES.md).

To modify it, you can use a configuration file, `.markdownlint.json`, which can be placed in the root of your repository.

For example:

```json
{
    "MD001": false,
    "MD013": false,
    "MD028": false,
    "MD029": false,
    "MD036": false,
    "MD033": {
        "allowed_elements": [
            "iframe",
            "url"
        ]
    }
}
```

This config file ignore certain rules defined by the tool, and allows specific elements that are not allowed by default.

__Markdown-link-check:__

```bash
npm install markdown-link-check
```

Checks for broken links in your Markdown files, ensuring all external references are valid. Note that this tools fails to check internal links from obvious reasons, therefore you can configure it to ignore them.

You can use `mlc_config.json` file to configure the tool, and place it in the root of your repository.

```json
{
    "ignorePatterns": [
        { 
            "pattern": "^https://github.com/myorganization/" 
        }
    ]
}
```

myorganization is the name of your organization in GitHub in this example.

__cspell:__

```bash
npm install cspell
```

Catches spelling mistakes, a critical aspect of maintaining professional and clear documentation. Although sometimes you might have words which aren't really a typo, but are not in the dictionary, therefore you can add them to the dictionary.

You can use `cspell.json` file to configure the tool, and place it in the root of your repository.

```json
{
    "words": [
        "tizen",
        "webos",
    ],
    "ignoreRegExpList": [
        "\\!\\[[^\\]]*\\]\$[^)]*\$",
        "(?<=```[\\s\\S]*?)\\S+",
        "\\[.*?\\]\$https?://[^\$]*\\)"
    ]
}
```

Note that I also added regex expression to ignore certain patterns, such as links, code blocks, and images alt text.


### Setting Up GitHub Actions Workflow

Integrating these tools into a GitHub Actions workflow ensures that every pull request adheres to your documentation standards. Here’s how you can configure the workflow for each tool:

```yaml
name: Documentation Checks

on:
  pull_request:
    paths:
      - 'docs'
      - 'docs/**'

jobs:
  markdown-lint:
    runs-on: ubuntu-latest
    timeout-minutes: 5

    steps:
    - name: Checkout
      uses: actions/checkout@v3
      with:
        fetch-depth: 0
  
    - name: Setup-env
      uses: ./.github/actions/setup-env
      with:
        node-version: '18.16'

    - name: Run markdownlint
      run: npm run lint

  link-checker:
    runs-on: ubuntu-latest
    timeout-minutes: 5

    steps:
    - name: Checkout
      uses: actions/checkout@v3
      with:
        fetch-depth: 0

    - name: Setup-env
      uses: ./.github/actions/setup-env
      with:
        node-version: '18.16'

    - name: Run markdown-link-check
      run: npm run link-check

  spell-checking:
    runs-on: ubuntu-latest
    timeout-minutes: 5

    steps:
    - name: Checkout
      uses: actions/checkout@v3
      with:
        fetch-depth: 0

    - name: Setup-env
      uses: ./.github/actions/setup-env
      with:
        node-version: '18.16'

    - name: Run cspell
      run: npm run spell-check
```

This is a simple workflow that runs on every pull request, and runs each linter separately.
In this case I specified the paths to the docs folder, but you can change it to your needs.

In the beginning of the workflow, I used a very simple composite github action to install node and npm, and then run the linters so i won't have to duplicate the same steps in each job. It's a simple action, but it's very useful, you can improve it to your needs.

If your'e interested in the composite action, here it is:

```yaml
name: 'Setup Environment'
description: 'Checkout and setup Node.js'
inputs:
  node-version:
    description: 'NodeJS version'
    required: true
    default: '18.16'

runs:
  using: "composite"
  steps:

    - name: Setup Node
      uses: actions/setup-node@v3
      with:
        node-version: ${{ inputs.node-version }}
    
    - name: install dependencies
      shell: bash
      run: npm install
```

![markdown-linters-workflow](/images/markdown-workflows.jpeg)

By integrating these Markdown linters into your GitHub Actions workflow, you elevate the standard of your project's documentation. This setup not only ensures consistency and correctness but also encourages a culture of quality and attention to detail within your development team. Remember, good documentation reflects the quality of your code and your commitment as a developer.


Do you have a repo that contains important documentation? Streamline your documentation with Markdown linters to enhance documentation quality.

Markdown Linters for Your Docs Repo


In continuation to my [previous post](./hosted-machines-monitoring), I took the OpenTelemetry Collector Contrib project and wrapped it in a minimal CLI using the [Appeal library](https://github.com/larryhastings/appeal/). This CLI is also compatible for all the common OS's (Windows, MacOS and Linux).

The goal was to make the OTEL binary a little bit more focused for our team use case, and easy for installation and usage.

## So What is Appeal?

Appeal is a command-line argument processing library for Python, like argparse, optparse, getopt, docopt, Typer, and click. But Appeal takes a refreshing new approach - the easiest approach I've seen so far. You're basically write Python functions, and Appeal translates them into command-line options and arguments.

Although its official documentation declares it wasn't tested on Windows yet, I found it to be working just fine (on Windows 10).


## My Use Case

I wanted to create a CLI that will be able to download the latest OTEL binary from the official GitHub repository, install it, run it, stop it and uninstall it without the user to worry about anything.
So in my use case at work I created the following commands: Download, Install, Run, Stop, Uninstall and Help is being generated for us automatically due to appeal's nature.

To start the CLI, I created `otel_cli` with a shebang so it'll be able to run as a script.
In addition to that i've added the imports used, initialization of the Appeal object and some global variables that will be used later.

```python
#!/usr/bin/env python3

import os
import shutil
import signal
import socket
import tarfile
import subprocess
import appeal
import ssl
import platform
import sys

from urllib.request import urlretrieve

app = appeal.Appeal()

ssl._create_default_https_context = ssl._create_unverified_context

CORALOGIX_DOMAIN = 'somedomain.coralogix.com'
CORALOGIX_APP_NAME = 'CI-Machine'
OTEL_PID = 'otelcol.pid'
_OTEL_CONTRIB_DIR = 'otel-contrib-dir'

#...commands...#

if __name__ == '__main__':
    app.main()

```

Now let's dig in into the commands themselves.

__download command__

```python
def get_machine_type():
    arch = platform.machine()
    if arch in ['AMD64', 'x86_64']:
        return "amd64"
    elif arch in ['arm64', 'aarch64']:
        return "arm64"
    else:
        print("Failed to detect architecture")
        raise ValueError(f"Unsupported architecture: {arch}")


def get_os_type():
    if platform.system() == 'Darwin':
        print("Detected MacOS")
        return "darwin"
    elif platform.system() == 'Linux':
        print("Detected Linux OS")
        return "linux"
    elif platform.system() == 'Windows':
        print("Detected Windows OS")
        return "windows"
    else:
        raise ValueError(f"Unsupported OS: {platform.system()}")

@app.command()
def download(version="0.89.0", ignore_cache=True):
    '''Download the OpenTelemetry Collector binary for the current machine.'''
    machine_type = get_machine_type()
    os_type = get_os_type()
    filename = f"otelcol-contrib_{version}_{os_type}_{machine_type}.tar.gz"
    url = f"https://github.com/open-telemetry/opentelemetry-collector-releases/releases/download/v{version}/{filename}"
    print(f"Downloading: {filename}...")
    if not os.path.exists(_OTEL_CONTRIB_DIR):
        os.mkdir(_OTEL_CONTRIB_DIR)
    urlretrieve(url, f'./{_OTEL_CONTRIB_DIR}/{filename}')
    print("Download completed.")

```

The `download` command is designed to fetch the OpenTelemetry Collector binary that matches the user's system architecture and operating system. This command leverages Python's built-in `platform` and `urllib` libraries to determine the system specifics and retrieve the necessary files.

`get_machine_type`: This function uses `platform.machine()` to identify the system's architecture. It supports both amd64 and arm64 architectures. If the architecture is not recognized, it raises a ValueError.

`get_os_type`: This function uses `platform.system()` to determine the operating system (Darwin for macOS, Linux, and Windows). Like `get_machine_type`, it raises a ValueError if the OS is not supported.

The download function first retrieves the machine type and OS type using the above functions. It constructs the filename for the binary to download, using the version number, OS type, and machine type.

The binary's URL is generated based on the filename. It checks if the directory `_OTEL_CONTRIB_DIR` exists and creates it if it doesn't. The binary is downloaded using `urlretrieve` and stored in `_OTEL_CONTRIB_DIR`. And eventually the function prints a message to indicate the completion of the download.


__install command__

```python
@app.command()
def install():
    '''Install the OpenTelemetry Collector binary for the current machine.'''
    files = [f for f in os.listdir(f'./{_OTEL_CONTRIB_DIR}') if f.endswith('.tar.gz')]
    if not files:
        raise FileNotFoundError("No tar.gz file found for installation.")
    tar_file = f'./{_OTEL_CONTRIB_DIR}/{files[0]}'
    print(tar_file)

    print(f"Installing {tar_file}...")
    with tarfile.open(tar_file, 'r:gz') as tar:
        tar.extractall(path=f'./{_OTEL_CONTRIB_DIR}')
    print("Installation completed. You can now run the application by running: otel_cli run")
```

The `install` command is responsible for setting up the OpenTelemetry Collector binary on the user's system. This command assumes that the binary has been downloaded using the download command and is present in a specific directory.

The function begins by searching for `.tar.gz` file within the `_OTEL_CONTRIB_DIR` directory. This is where the downloaded binary should be located. If no such file is found, it raises a FileNotFoundError.

Once the file is located, the script prints the path of the tar file, indicating the binary to be installed. Using Python's `tarfile` module, the script opens the .tar.gz file in read mode and extracts its contents to _OTEL_CONTRIB_DIR.

After successfully extracting the files, the script prints a message indicating that the installation is complete and that the user can now run the application.

The use of `tarfile.open` in the with statement ensures that the tar file is correctly opened and closed after the operation, following Python's recommended practice for file handling. The `extractall` method is used to extract all contents of the tar file into the specified directory. This approach abstracts the extraction process from the user, making the installation process straightforward and user-friendly.


__run command__

```python
@app.command()
def run(config_file="config.yaml"):
    '''Run the OpenTelemetry Collector binary for the current machine.'''
    binary_name = 'otelcol-contrib'
    if sys.platform == 'win32':
        binary_name += '.exe'

    otel_col_path = os.path.join(os.getcwd(), _OTEL_CONTRIB_DIR, binary_name)
    if not os.path.isfile(otel_col_path):
        raise FileNotFoundError(f"{otel_col_path} binary not found. Please install first.")

    os.environ['CORALOGIX_SUBSYS_NAME'] = socket.gethostname()

    if os.path.isfile('.env'):
        with open('.env') as f:
            for line in f:
                key, value = line.strip().split('=', 1)
                os.environ[key] = value

    os.environ['CORALOGIX_DOMAIN'] = CORALOGIX_DOMAIN
    os.environ['CORALOGIX_APP_NAME'] = CORALOGIX_APP_NAME

    print("Starting OpenTelemetry Collector...")
    process = subprocess.Popen([os.path.join(os.getcwd(), _OTEL_CONTRIB_DIR, 'otelcol-contrib'), '--config', config_file])
    with open(OTEL_PID, 'w') as otel_pid:
        otel_pid.write(str(process.pid))
    print(
        f"OpenTelemetry Collector is running in the background with APP_NAME: "
        f"{os.environ['CORALOGIX_APP_NAME']} and Subsystem: "
        f"{os.environ['CORALOGIX_SUBSYS_NAME']}")
```

The `run` command in the CLI tool is designed to start the OpenTelemetry Collector with a specified configuration. This command is crucial as it activates the collector, allowing it to begin its telemetry data collection and processing.

The function starts by setting the default binary name to `otelcol-contrib`.
It then checks if the platform is Windows `(sys.platform == 'win32')`. For Windows, it appends .exe to the binary name, accommodating the typical executable format in Windows.

The script constructs the full path to the binary using `os.path.join`, combining the current working directory, the _OTEL_CONTRIB_DIR, and the binary name.
If the binary does not exist at the constructed path, a FileNotFoundError is raised, prompting the user to install the binary first.

The script sets the `CORALOGIX_SUBSYS_NAME` environment variable to the hostname of the current machine, using `socket.gethostname()` (this is how the machine your'e monitoring will displayed in your future Coralogix dashboards).
It then reads environment variables from a `.env` file, if present, and sets them in the current environment.
The `CORALOGIX_DOMAIN` and `CORALOGIX_APP_NAME` are set to a predefined values (global variables above).

The script prints a message indicating the start of the OpenTelemetry Collector.
It uses `subprocess.Popen` to start the OpenTelemetry Collector in the background, passing the path to the binary and the configuration file as arguments.
The process ID (PID) of the started collector is written to a file named `OTEL_PID`. This PID is used later for stopping the collector.

After successfully starting the collector, the script prints a message with details about the running instance, including the APP_NAME and Subsystem, derived from the environment variables.

__stop command__

```python
@app.command()
def stop():
    '''Stop the OpenTelemetry Collector.'''
    if not os.path.exists(OTEL_PID):
        print("OpenTelemetry Collector is not running or pid file is missing.")
        return

    with open(OTEL_PID, 'r') as otel_pid:
        pid = int(otel_pid.read().strip())

    try:
        if sys.platform == 'win32':
            subprocess.run(['taskkill', '/F', '/PID', str(pid)], check=True)
        else:
            os.kill(pid, signal.SIGTERM)

        os.remove(OTEL_PID)
        print("OpenTelemetry Collector has been stopped.")
    except ProcessLookupError:
        print("Process not found. It may have already been stopped.")
    except Exception as e:
        print(f"Error stopping the process: {e}")
```

The `stop` command in the CLI tool is designed to safely terminate the OpenTelemetry Collector that's running in the background. This command is crucial for stopping the collector's processes and ensuring that resources are properly released.

The function first checks if the `OTEL_PID` file exists. This file contains the process ID (PID) of the running OpenTelemetry Collector. If the file does not exist, it indicates that the collector is not running, and the function returns after printing an appropriate message.

To read the file with the PID in it, for Windows systems `(sys.platform == 'win32')`, it uses the `subprocess.run` method to execute the `taskkill` command, forcibly terminating the process with the specified PID.
For other operating systems, it uses the `os.kill` function with the SIGTERM signal to terminate the process. This is a more Unix/Linux traditional way of handling process termination.

The script then removes the `OTEL_PID` file, signifying that the collector has been stopped.

The script also includes exception handling for situations where the process may not be found (ProcessLookupError) or other unexpected errors occur. This ensures that any issues during the stopping process are gracefully caught and reported to the user.

Eventually, the script prints a message to inform the user that the OpenTelemetry Collector has been stopped successfully, providing clear feedback on the action's outcome.

__uninstall command__

```python
@app.command()
def uninstall():
    '''Uninstalls the OpenTelemetry Collector files and binaries.'''
    if os.path.exists(_OTEL_CONTRIB_DIR):
        print(f"Uninstalling OpenTelemetry Collector from {_OTEL_CONTRIB_DIR}...")
        shutil.rmtree(_OTEL_CONTRIB_DIR)
        print("Un-installation complete. OpenTelemetry Collector has been removed.")
    else:
        print("OpenTelemetry Collector is not installed or has already been uninstalled.")
```

The `uninstall` command is a crucial component of the CLI tool, enabling users to remove the OpenTelemetry Collector and its associated files from their system. This command ensures that users can cleanly and easily reverse the installation process.

The command starts by checking if the `_OTEL_CONTRIB_DIR` directory exists. This directory is where the OpenTelemetry Collector's binaries and related files are stored post-installation. If the directory exists, indicating that the collector is installed, the script prints a message to inform the user that the uninstallation process has begun.

It then uses the `shutil.rmtree` function to recursively delete the _OTEL_CONTRIB_DIR directory, thereby removing all the files and binaries related to the OpenTelemetry Collector. This function is a powerful tool in Python for directory removal, capable of deleting a directory and all its contents.
After successful removal of the directory, the script prints a confirmation message indicating the completion of the uninstallation process.

If the _OTEL_CONTRIB_DIR directory does not exist, the script assumes that the OpenTelemetry Collector was never installed or has already been uninstalled. In such cases, it prints a message stating that the OpenTelemetry Collector is not installed or has already been removed.
This check prevents the script from attempting to delete non-existent directories, which could lead to errors.

## Usage

For convenience, add the `otel_cli` executable to your PATH and update its symlink to point the python binary. You can do that by running the following command (MacOS):

```bash
ln -s $PWD/otel_cli /usr/local/bin/otel_cli
```

To download the OpenTelemetry collector, run the following command:

```bash
otel_cli download
```

Then you can install the OpenTelemetry collector as a service by running the following command:

```bash
otel_cli install
```

You can start the OpenTelemetry collector by running the following command:

```bash
otel_cli run
```

You can stop it by running the following command:

```bash
otel_cli stop
```

You can uninstall it by running the following command:

```bash
otel_cli uninstall
```

For further help simply run:

```bash
otel_cli help
```

And it'll output all the available commands and their descriptions (taken from the functions docstring).


In continuation to my previous post, I'll show you how to create a minimal and compatible CLI using the Appeal library.

Easy Minimal CLI Using Appeal Library


I encountered this task at work where I needed to monitor a local machines that are considered critical for us in terms of tests runners and CI. 
The machines were running a few services that required some effective monitoring and alerting.

Recently, our team has transitioned from NewRelic to Coralogix for monitoring, and integrating it with OpenTelemetry Collector Contrib was done for all of our local machines. This post details the steps involved.

After a short research, I found out that OpenTelemetry Collector Contrib has a [hostmetrics receiver] that can be also used to monitor local machines and send the data directly to our new vendor - Coralogix.

OpenTelemetry provides a single set of APIs and libraries that standardize how you collect and transfer telemetry data. You can use these APIs and libraries to instrument your code and send telemetry data to a backend of your choice. The OpenTelemetry Collector offers a vendor-agnostic implementation on how to receive, process, and export telemetry data. In addition, it removes the need to run, operate, and maintain multiple agents/collectors in order to support open-source observability data formats (e.g. Prometheus, etc.) sending to multiple open-source or commercial back-ends.

__So let's start__

Download the otelcol-contrib binary for your platform from OpenTelemetry [GitHub](https://github.com/open-telemetry/opentelemetry-collector-releases/releases).

If you're using Coralogix as well, you'll need to obtain your Coralogix Private Key and set it along with other required environment variables:

```bash
CORALOGIX_DOMAIN
CORALOGIX_APP_NAME
CORALOGIX_SUBSYS_NAME
CORALOGIX_PRIVATE_KEY
```
Create a `config.yaml` file, including necessary metrics, scrapers, log level, and intervals. Refer to [OpenTelemetry's Host Metrics Receiver](https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/receiver/hostmetricsreceiver) for further information.

Add coralogix as an exporter in the config, as shown below as an example:

```yaml
receivers:
  hostmetrics:
    collection_interval: 30s
    scrapers:
      network:
      cpu:
      memory:
      disk:
exporters:
  coralogix:
    traces:
      endpoint: "otel-traces.${CORALOGIX_DOMAIN}:443"
    metrics:
      endpoint: "otel-metrics.${CORALOGIX_DOMAIN}:443"
    logs:
      endpoint: "otel-logs.${CORALOGIX_DOMAIN}:443"
    domain: "$CORALOGIX_DOMAIN"
    private_key: "$CORALOGIX_PRIVATE_KEY"
    application_name: "$CORALOGIX_APP_NAME"
    subsystem_name: "$CORALOGIX_SUBSYS_NAME"
    application_name_attributes:
      - "service.namespace"
    subsystem_name_attributes:
      - "log.file.name"
      - "service.name"
    timeout: 30s
  logging:
    verbosity: normal
service:
  pipelines:
    metrics:
      receivers: 
       - hostmetrics
      exporters: 
       - coralogix
```

Execute the binary with the config file flag: 

```bash
./otelcol-contrib --config=config.yaml
```

And there you go. You should be able to see the metrics in your metrics vendor exploration by the subsystem name or application name.

![Coralogix-Widget](/images/coralogix-widget.png)


Learn how to seamlessly integrate OpenTelemetry Collector Contrib with Coralogix for efficient monitoring of local hosted machines.

Welcome to
Kazi's Dev Blog

Newest Blog Posts(See All)

Integrating Azure AD Authentication in Your Next.js App Using MSAL

Scheduling Lambdas Using AWS EventBridge and Terraform

CORS - Introduction For The New FE/BE Developer

GitHub Actions for Dynamic Cross-Platform Testing

Why Makefile is One of My Essentials Tools in Every Software Project

Markdown Linters for Your Docs Repo

Easy Minimal CLI Using Appeal Library

Monitor Your Hosted Local Machines with OpenTelemetry Collector Contrib

Explore More Blog Posts

Welcome to Kazi's Dev Blog

Newest Blog Posts(See All)

Integrating Azure AD Authentication in Your Next.js App Using MSAL

Scheduling Lambdas Using AWS EventBridge and Terraform

CORS - Introduction For The New FE/BE Developer

GitHub Actions for Dynamic Cross-Platform Testing

Why Makefile is One of My Essentials Tools in Every Software Project

Markdown Linters for Your Docs Repo

Easy Minimal CLI Using Appeal Library

Monitor Your Hosted Local Machines with OpenTelemetry Collector Contrib

Explore More Blog Posts

Welcome to
Kazi's Dev Blog