Adding a Custom DNS Entry to CoreDNS

I ran into a small problem recently when I was leveraging my site updating code referenced in Automating Static WordPress Updates. The problem was that I was unable to update content reliably for two reasons:

  1. The content was not properly switching out the hostname in the URL when I would crawl my backend WordPress site. I actually implemented something that helped to correct this but it lead to problem #2. I should probably post a new article on the changes I made in my script…
  2. My script would only crawl the external static site so updates were not getting published. This lead me to creating this post!

Now that I have the problems covered, let’s get right to it. In order to resolve the issue, I needed my Kubernetes to have split DNS for certain hosts. I needed my static site updating script to be able to crawl my backend WordPress and NOT crawl the public facing static site.

Edit coreDNS’s configmap

In order to add a custom entry to your Kubernetes, you can simply edit coredns’s configmap and add a new hosts entry. Here is my current coredns configmap:

apiVersion: v1
data:
  Corefile: |
    .:53 {
        errors
        health
        ready
        kubernetes cluster.local in-addr.arpa ip6.arpa {
          pods insecure
          fallthrough in-addr.arpa ip6.arpa
        }
        prometheus :9153
        forward . /etc/resolv.conf
        cache 30
        loop
        reload
        loadbalance
        import custom/*.override
    }
    import custom/*.server
kind: ConfigMap

Based upon the hosts information provided by CoreDNS, we just add a new host block and life will be good, right? I noticed that in the documentation, I see This plugin only supports A, AAAA, and PTR records. That’s not going to work since I’m wanting to point to a new hostname. Instead, we’ll use the rewrite syntax. Below is my updated CoreDNS configmap.

apiVersion: v1
data:
  Corefile: ".:53 {
    errors
    health
    rewrite stop {
       name exact live-blog.shellnetsecurity.com. nginx-npp.wordpress.svc.cluster.local
       answer name nginx-npp.wordpress.svc.cluster.local. live-blog.shellnetsecurity.com
    }   
    ready
    kubernetes cluster.local in-addr.arpa ip6.arpa {
      pods insecure
      fallthrough in-addr.arpa ip6.arpa
    }
    prometheus :9153
    forward . /etc/resolv.conf
    cache 30
    loop
    reload
    loadbalance
    import
    custom/*.override
  }
  import custom/*.server"
kind: ConfigMap

Fixed and no restart required like with using hosts

Oracle Cloud Vault KMS – Replicating AWS Secrets

I recently had to do some tinkering in Oracle Cloud Infrastructure (OCI) with Compute Instances and the Key and Secrets Management in Oracle Vault. I wanted to be able to replicate AWS EC2 instance capability to access secrets based upon the instance’s permissions without requiring any usernames, passwords, tokens, etc… to be stored on the instance itself. In AWS, I already know how to do this via IAM Roles and Policies. Before we begin, I’m assuming that you already have created a Vault and Secret that will be accessed. You can reference the Oracle Cloud documentation on creating secrets and vaults here. I also assume that you have a Compute instance created that is running some type of Linux. Mine is running OL8.

The OCI documentation does a great job at pointing you to interacting with secrets via CLI, REST API, and console. I struggled to find clear instructions on allowing a compute instance to access secrets until I stumbled upon Dynamic Groups. A Dynamic Group can be referenced in policies and OCI resources can be added as members. This is what I needed!

Creating a Dynamic Group

This part was pretty easy. Within the Identity section of the console, you can find Dynamic Groups.

I created one that was called ComputeGroup. When creating a Dynamic Group, you need to create rules that determine what OCI resources will be members of the group. OCI refers to these rules as Matching Rules. The documentation states that the following are supported variables for rules:

  • instance.compartment.id – the OCID of the compartment where the instance resides
  • instance.id – the OCID of the instance
  • tag.<tagnamespace>.<tagkey>.value – the tag namespace and tag key. For example, tag.department.operations.value.
  • tag.<tagnamespace>.<tagkey>.value='<tagvalue>' – the tag namespace, tag key, and tag value. For example, tag.department.operations.value='45'

I decided to make my rule flexible by using a tag like the below:

Any {tag.permissions.getSecrets.value = 'true'}

You can find some additional details regarding tags and namespaces in the OCI documentation here.

Updating Secret Policies

Now that I had a Dynamic Group created, I needed to allow that group in the secrets policy. Policies are also located under Identity.

I already had an existing policy called AccessSecrets that I was using to allow certain users to access secrets via API calls. I used the Let users read, update, and rotate all secrets example to create my first statement in the policy.

Allow group KeyReaders to use secret-family in tenancy	

This statement allows members of my KeyReaders group to be able to access secrets. I needed a new statement that allowed my Dynamic Group access to the secrets as well. I added the below statement to my policy.

Allow dynamic-group ComputeGroup to use secret-family in tenancy	

Adding a Tag to an Instance

The next step is to apply the tag to whatever instance I’d like to allow access to my secret. You can do this by opening the instance details and clicking the Add Tags item from the More Actions menu. Select the tag and value specified in the policy.

Once you have your tag, click the Add Tags button and we’re ready to move on.

Example Code

In order to make life easier, I created my example code based upon the example code located here in OCI’s Python SDK examples. This example makes use of a config file and would expect an API token in order to connect. We’re not doing that so I made the below changes to their example:

$ diff secretclient_example.py my_secretclient_example.py 
32c32
< if len(sys.argv) != 3:
---
> if len(sys.argv) != 2:
37d36
< oci_profile = sys.argv[2]
39,41c38
< config = config = oci.config.from_file(
<     "~/.oci/config",
<     oci_profile)
---
> signer = oci.auth.signers.InstancePrincipalsSecurityTokenSigner()
43c40
< secret_client = oci.secrets.SecretsClient(config)
---
> secret_client = oci.secrets.SecretsClient(config={}, signer=signer)

This script expects the OCID of the secret to be provided as an argument and then run the command to get the secret’s details:

$ python my_secretclient_example.py ocid1.vaultsecret...a
Reading vaule of secret_id ocid1.vaultsecret...a.
Decoded content of the secret is: {
  "username" : "admin",
  "password" : "Password1!"
}.

Additional Reference

After going through all of this and having difficulty navigating the documentation. I stumbled upon this Oracle document :facepalm: that also provided just about all of the details that I needed. Like the title of the blog states, these are just My Battles With Technology.

Checking and Correcting Permissions in Amazon Redshift

I needed to make sure I had two users in an Amazon Redshift cluster (readonly and readwrite). I needed to make sure their permissions were set appropriately so the first step was to first see what their permissions were on the schema and then the tables.

First we check their schema permissions with the below query that makes use of the has_schema_privilege function:

SELECT
    u.usename,
    s.schemaname,
    has_schema_privilege(u.usename,s.schemaname,'create') AS user_has_create_permission,
    has_schema_privilege(u.usename,s.schemaname,'usage') AS user_has_usage_permission
FROM
    pg_user u
CROSS JOIN
    (SELECT DISTINCT schemaname FROM pg_tables) s
WHERE
    (u.usename = 'readonly'
    OR
    u.usename = 'readwrite')
    AND
    s.schemaname = 'public';

The query above filters on the specific users and schema I’m checking against. You are welcome to customize this query to look for all users and/or all schemas. This query results in the below permissions on the public schema for these users:

   usename  | schemaname | user_has_create_permission | user_has_usage_permission 
-----------+------------+----------------------------+---------------------------
 readonly  | public     | t                          | t
 readwrite | public     | t                          | t

This looks good so let’s check the tables within the schema using the has_table_privilege function:

SELECT
    u.usename,
    t.schemaname||'.'||t.tablename as path,
    has_table_privilege(u.usename,t.tablename,'select') AS has_select,
    has_table_privilege(u.usename,t.tablename,'insert') AS has_insert,
    has_table_privilege(u.usename,t.tablename,'update') AS has_update,
    has_table_privilege(u.usename,t.tablename,'delete') AS has_delete,
    has_table_privilege(u.usename,t.tablename,'references') AS has_reference
FROM    pg_user u
CROSS JOIN
    pg_tables t
WHERE
    (u.usename = 'readonly'
    OR
    u.usename = 'readwrite')
    AND
    t.schemaname = 'public'
    AND
    t.tablename = 'employees';

Again, I’m querying for very specific details. You are welcome to alter the query to fit your needs. This query results in the following:

  usename  |      path       | has_select | has_insert | has_update | has_delete | has_reference
-----------+-----------------+------------+------------+------------+------------+--------------
 readonly  | public.employees | f          | f          | f          | f          | f
 readwrite | public.employees | f          | f          | f          | f          | f

It looks like these users do not have the correct access to the public.employees schema and table so this needs to be corrected. You can reference the AWS Redshift GRANT command details to see what permissions we should grant these users. I’ll use the below GRANT statements to correct each users permissions respectively.

GRANT SELECT ON ALL TABLES IN SCHEMA public TO readonly;
GRANT ALL PRIVILEGES ON ALL TABLES IN SCHEMA public TO readwrite;

Now let’s double check our work:

dev=# SELECT
    u.usename,
    t.schemaname||'.'||t.tablename as path,
    has_table_privilege(u.usename,t.tablename,'select') AS has_select,
    has_table_privilege(u.usename,t.tablename,'insert') AS has_insert,
    has_table_privilege(u.usename,t.tablename,'update') AS has_update,
    has_table_privilege(u.usename,t.tablename,'delete') AS has_delete,
    has_table_privilege(u.usename,t.tablename,'references') AS has_reference
FROM    pg_user u
CROSS JOIN
    pg_tables t
WHERE
    (u.usename = 'readonly'
    OR
    u.usename = 'readwrite')
    AND
    t.schemaname = 'public'
    AND
    t.tablename = 'employees';

  usename  |      path       | has_select | has_insert | has_update | has_delete | has_reference 
-----------+-----------------+------------+------------+------------+------------+--------------
 readonly  | public.employees | t          | f          | f          | f          | f
 readwrite | public.employees | t          | t          | t          | t          | t
(2 rows)

These permissions look like what we’d expect! We can further test with the user’s themselves. The below code block is attempting a select and then an update with the readonly user.

dev=> select * from employees limit 1;
 employee_id | employee_name | employee_status |       manager        |    employee_email     |     ssn     | vip | location | payment_method
------------+--------------+----------------+---------------------+----------------------+-------------+-----+----------+----------------
          1 | Bob Smith  | Inactive       | [email protected] | [email protected] | 802-85-6966 | f   | 1        | phpe
(1 row)

dev=> update employees set employee_email='[email protected]' where employee_id=1;
ERROR:  permission denied for relation employees

We can issue a SELECT but the UPDATE fails for a permissions issue so this looks good.

Let’s checkout our readwrite user:

dev=> select * from employees limit 1;
 employee_id | employee_name | employee_status |       manager        |    employee_email     |     ssn     | vip | location | payment_method
------------+--------------+----------------+---------------------+----------------------+-------------+-----+----------+----------------
          1 | Bob Smith  | Inactive       | [email protected] | [email protected] | 802-85-6966 | f   | 1        | phpe
(1 row)
dev=> update employees set employee_email='[email protected]' where employee_id=1;
UPDATE 1
dev=> select * from employees limit 1;
 employee_id | employee_name | employee_status |       manager        |    employee_email     |     ssn     | vip | location | payment_method
------------+-----------------+----------------+-----------------------+-----------------------------+-------------+-----+----------+----------------
          1 | Bob Smith  | Inactive       | [email protected] | [email protected] | 802-85-6966 | f   | 1        | phpe
(1 row)

Yaay it worked! Yes, this article assumes you already have the users created. Yes, we probably should look at doing role based permissions instead of giving the users direct permission. This was a small dev environment so I didn’t need to go crazy.