Steps to Install Python Control library (For LQR control) On Raspberry Pi

I got a Raspberry Pi 3 and would like to program python on it with LQR for stabilized control. Here are the steps that I took to install the Python control package and its dependencies:

pip3 install control --upgrade
sudo apt-get install build-essential
sudo apt-get install ninja-build
sudo apt install gfortran
pip3 install slycot

To test the installed control package, run the following snippet:

>>> import control
>>> control.lqr([[1]], [[1]],[[1]],[[1]])
(array([[2.41421356]]), array([[2.41421356]]), array([-1.4142135+0.j], dtype=complex64))

Run AWS Lambda Function Code Locally - Quick Tutorial

Why do I want to Run the Lambda Function Code locally?

One of my dev friends falsely believed that AWS Lambda Function would require deployment in order to run. This is not true. It is a common misunderstanding since in the Lambda Function you are only defining a handler.

The solution is simple - just write a main function that includes your lambda handler and pass parameters to it! No deployment needed.

How to Do it?

Here's your lambda function in index.js

exports.handler = (event, context, callback) => {
  console.log('hello', event.name, 'from', process.env.SERVICE_NAME);
  callback(undefined, 'done');
};

Write your main.js

process.env.SERVICE_NAME = 'commandline lambda function';

const index = require('./index.js');

const context = {};

const event = {
  "name": "world"
}

index.handler(event, context, (err, data) => {
  if (err) { console.error('error: ', err); }
  if (data) { console.log('data: ', data); }
});

You have environment variables defined, have your event defined, and invoked the lambda function handler. Now just call it from the terminal:

node main.js

Note that it will use the credential in your ~/.aws/credentials file, just like the aws-cli tool.

How to run the command line lambda function with aws-okta credentials:

aws-okta exec my-aws-account-name -- node main.js

Getting Started with Vim - Quick Tutorial

Why would you want to use Vim?

When you ssh into a server, you will need a text editor that works in the terminal to change for example, server configurations. Nano is a text editor that works on terminal. But it is hard to navigate fast with Nano. Improve your Vim skill for the speed!

How to use Vim?

To open a file:

• In the terminal, type in:  vi /tmp/1.txt

To navigate:

• Your cursor keys just work
• Press 'control' + 'D' for pageDown
• Press 'control' + 'U' for pageUp

To get to the bottom of the doc:

• Press 'shift' + 'G'

To get to the top of the doc:

• Type in ':0' then 'enter'

To get to the start of the line:

• Press 'shift' + '^'

To get to the end of the line:

• Press 'shift' + '$'

To get to the 50th line of the doc:

• Type in ':50' then 'enter'

To delete the current line:

• Type in 'dd'

To delete the next 20 lines:

• Type in '20dd'

To delete the all lines:

• Type in '100000dd'

• Repeat until there's no line left

To save file and exit:

• Type in ':wq!' then 'enter'

To exit without saving:

• Type in ':q!' then 'enter'

To edit:

• Press the 'i' key to enter editing mode

• Your 'delete' key and 'backspace' key just work

To exit editing mode:

• Press the 'esc' key

To undo a change:

• Press the 'esc' key to exit the edit mode

• Press the 'u' key to undo

To get to a word:

• Press the '/' key to search forward

• Example: '/hello' will search forward the next hello that appears in the text

• Press the 'n' key to get to the next 'hello'

To get to a word from the bottom of the doc:

• Press 'shift' + 'G' to get to the bottom

• Press 'shift' + '?' key to search backward

• Example: '?hello' will search last hello that appears in the text

• Press the 'n' key to get to the next 'hello'

To delete a word:

• Use the above method to get to the word

• Press 'dw' to delete the word

To delete one character:

• Use the above method to get to the word

• Press 'x' to delete a character

To copy text: 

• Your cursor just works in selecting text
• Your Windows command just works: 'ctrl' + 'c'
• 'cmd' + 'c' for OSX

To paste text:

• Press the 'i' key to enter editing mode
• Your Windows command just works: 'ctrl' + 'v'
• 'cmd' + 'v' for OSX

 If your terminal does not support text selection with copy & paste, try:

To copy 1 line:

• Press 'yy' to copy the current line

To paste 1 line:

• Press 'p' to paste a copied a line

Great. You know how to use vim.

Delay Processing of a List Using JavaScript Promise without Recursion

In many cases, we'd like to perform a list of actions with delay in between. (Ex: to make a list of API calls without overloading the server.) However, JavaScript has no sleep function, but it is provided with a setTimeout method callback after a delay. Typically tail recursion can be applied at the end of function to trigger another setTimeout for the next execution. However this makes code ugly and can be avoided by using Promise. The setTimeout function can be easily wrapped into a promise to provide delay by:

function promiseDelay(miliseconds) { return new Promise((resolve) => { setTimeout(resolve, miliseconds)})}

To use this function to delay your call, just place the logic inside .then() method

promiseDelay(1000).then(() => {console.log('hello')})

When there's a list of values to be processed, we can keep chaining new deplayPromise at the end of the promise:

p = Promise.resolve(undefined)
[1,2,3].forEach(x=> {
  p = p.then(() => promiseDelay(1000).then(() => {console.log(x)}) )
})

This is a little hard to read, as there is .then() inside a .then(). Note that the next execution is not depending on the result of the delay, this is equivalent to wrap the execution into another promise, and use Promise.all() to ensure both delay and execution to be both completed before proceeding:

function promiseDisplay(x) {
  return new Promise((resolve) => { console.log(x); resolve()})
}

Promise.all([promiseDelay(1000), promiseDisplay(x)]);

Applying the above change, the code now looks nice:

Dealing with Connection Timeout to Redis when the "Reclaimed Items" Spike

Recently I encountered a Redis problem, where new Redis connections failed to be established from the server, and the client massively timed out. The AWS managed Redis cluster has a few metrics coincide with the surge of connection issue: "New Connections" and "Reclaimed Items" both spiked during the time of difficulty.

Commonly Redis would respond slow, causing connection issue on the client when "Eviction" events occurs. This usually happens where the max capacity of the Redis node has been reached, and entries are forced out of Redis as a result. However, in my case the "Eviction" metrics showed 0, and "Bytes Used for Cache" suggested that Redis has enough RAM. Clearly, the surge of "New Connections" is a symptom rather than a direct cause. When the system was halting, normally connections would be expected to build up waiting on a certain resource.

The AWS documentation shows that the "Reclaimed Item" is the
"The total number of key expiration event". https://docs.aws.amazon.com/AmazonElastiCache/latest/red-ug/CacheMetrics.Redis.html

Redis documentation explains that "basically [key] expired events are generated when the Redis server deletes the key" after TTL on a key goes to zero. https://redis.io/topics/notifications Since the background Redis process may keep deleting items when the collected sample has >25% of expired item, Redis may take a few seconds to delete when large amount of items are expiring.  https://redis.io/commands/expire This would make established connections to be unable to proceed their operations, causing connection issue for the client.

Note that this problem is related to both write/second and the TTL length. You have an option of either reducing write/second, or make TTL shorter so that the number of items don't build up at a potential of being deleted all at once by the background process.