Sunday, October 09, 2022

How to run AR.js basic example

Git clone the project of https://github.com/AR-js-org/AR.js

The example has to run from an http server. Opening the example file in the directory won't work. Install nodejs http server

    npm install http-server -g

In your terminal, change directory to the root of AR.js, and run

    http-server

    The server is serving the html files in http://127.0.0.1:8080. Get to http://127.0.0.1:8080/three.js/examples/basic.html in your browser. Allow the page to access camera. The page will start recording you. On your phone, do a Google image search for "hiro marker".  Display the marker on your phone and place in front of the camera. The polygon animation will render on the marker.



Friday, July 22, 2022

AWS Java SDK DynamoDBv2 Scan

AWS Java SDK DynamoDBv2 (com.amazonaws.services.dynamodbv2.document.Table) has a terrible API at performing scan operation. Against common sense, the table.scan(scanSpec) returns a ItemCollection object, which requires the developer to call ItemCollection.iterator() in order to trigger an actual scan. If the ItemCollection.iterator() method is not triggered, the itemCollection.lastLowLevelResult field will be null.

This doesn't work, and will reach Null Pointer Exception:

itemCollection = table.scan(scanSpec) 

System.out.println(itemCollection.lastLowLevelResult.items.size)

This will work - calling of iterator method is required to populate the itemCollection.lastLowLevelResult field.

itemCollection = table.scan(scanSpec)

          List<Item> items = new ArrayList() 

CollectionUtils.addAll(items, itemCollection.iterator()) 

System.out.println(itemCollection.lastLowLevelResult.items.size)

Monday, July 11, 2022

XGBoost Parameter

This is a quick documentation of my understanding of the XGBoost parameters

  • max_depth: how deep can one tree grow

  • num_rounds : how many trees are in a prediction model

  • learning_rate: the weight between applying result (residual value) to the next tree

  • alpha: regularization term. (related to pruning trees)

  • lambda: regularization term. (related to pruning trees)

  • gamma: minimum loss reduction (related to limiting the depth of a tree)

  • Reference: 

Saturday, June 11, 2022

Java: wait & notify, await & signal, park & unpark

There are several way to stop a thread in Java, to get awaken later. Here are their usages:

wait & notify

Every object has a .wait() and .notify() method. These methods must be called in a synchronized block. 

When a .notify() happened before .wait(), it will not awake the thread.

await & signal 

With ReentrantLock, a condition object can be pulled from the lock, by `lock.newCondition()`. When the lock is locked, .await() These methods needs to be call when the lock is in lock state.

When a .signal() happened before .await(), it will not awake the thread.

LockSupport: park & unpark

Unlike wait & await, LockSupport.park() and LockSupport.unpark(t) and doesn't need to be in a locked / synchronized block. Since LockSupport is permit based, unpark assign a permit to a thread, which can be later used in park . So the order of park and unpark is not strict. Notice that permit doesn't have a counter - it can only be used in 1 park call.


Thursday, June 09, 2022

Java Locks: synchronized. ReentrantLock, ReentrantReadWriteLock, StampedLock

synchronized vs. ReentrantLock

Both create critical sections. ReentrantLock is unstructured and can lock and unlock in different methods. ReentrantLock can tryLock with a timeout.

ReentrantLock vs ReentrantReadWriteLock

ReentrantLock creates a critical sections that blocks both read & write. ReentrantReadWriteLock allows readLocks to read together, while blocking by critical sections when write is involved. Note that since readLock can block writeLock, it could result in writeLock starvation when a lot of readLocks appear. WriteLock cannot proceed until all readLocks are unlocked.

ReentrantReadWriteLock vs. StampedLock

ReentrantReadWriteLock is a pessimistic lock, which doesn't read while writer writes, and stops writer when it reads.

StampedLock is an optimistic lock, which reads (by tryOptimisticRead) while allowing write to happen, but also detects write (by validate) - if write happens during the time, simply reads again. 


Thursday, February 24, 2022

Getting Started with Haskell - Quick Tutorial

 To install Haskell compiler on MacOS:

brew install ghc

To compile a Haskell file into executable: (the Haskell file must have a main function)

ghc my-haskell-with-main-func.hs

To run without a Haskell file compiling: (the Haskell file will execute without a main function)

ghc -e ':script my-haskell-without-main-func-each-line-is-a-command.hs'

Haskell interactive console:

ghci

To output a string

putStrLn "hello"

To create a Hello World Haskell program

main = putStrLn "hello"

To create a multi-line Haskell program

  main = do

    putStrLn "hello"

    putStrLn "World"

To get stdin to a variable

t <- getLine 

To cast a String value into Int

i = read "123" :: Int

To covert a Int to String

s = show i

To round Float to Int

i = round  1.5

To split a String to a List by space

wordList = words "a b c"

To get item by index from a List 

wordList !! 2

To split a String to a List by new lines

 lineList = lines "hello world\nhi there"

To print a variable

print "hello"

To create a list of integer sequence from 1 to 5

[1..5]

To create a list of integers

[1,2,3] 

To define a lambda function

foo = \x -> x+1

To call a lambda function

foo 1 

To map a List  and apply a lambda function

map (\x -> x+1) [1 .. 2]

To filter a List  and apply a lambda function

filter (\x -> x>0) [1,2,3,0,5,6]

To take unique values of a list

import Data.List

nub [1,2,2,3,3,4]

To sort a List

sort [2,3,1] 

 To compose functions, use .

(map (\x -> x+1) . nub) [1,2,2,2,2]

To pipe functions from right to left without compose, use $

map (\x -> x+1) $ nub [1,2,2,2,2]

To flatten / merge multiple Lists into one List

concat [[1,2],[3,4]]

To join a List of String to one String with a delimiter

intercalate " " ["a", "b", "c"]  

To reduce, use foldr

foldr (\x s -> s+x) 100 [1,2,3]

foldr is kind of slow. Use foldl

foldl (\s v -> s+v) 100 [1,2,3] 

To sum numbers in a List

sum [1,2,3] 

To divide a List into a List of groups (Lists), each of which collects the repeated items in sequence:

groupBy (==) "aabbcccdddde" 

Write a function that returns constant value

f = (const 1)

f 100

To get combination of of a list with itself (Note: a String is a List)

        mapM (const "ABC") [1,1] 

To comment

-- This is a comment

To take power of a number

x ** 2     -- will result in Float value

x ^ 2       -- will result in Int value

To take absolute value

abs x

To take modulo of two values (% doesn't work)

mod 5 3 

To perform integer division

div 5 3 

To compare values with not equals (!= doesn't work)

1 /= x 

To get the max value in a List of values

maximum [1,2,3] 

To concat two Strings

"Hello" ++ "World"

To concat two Lists

[1,2,3] ++ [4,5,6]

To reverse a List

reverse [1,2,3] 

To take a for loop (

mapM_ (\i -> do {

    print $ "hello" ++ show i

})  [1.. 5]

-- Note: what do {} is used, remember to add ; at the end of each line, except for the last line.

-- Note: use mapM_ for side effect, and expect mapM_ to return no value.

To use ternary operator

x = 2

y = if x > 1 then "YES" else "NO" 

To define a variable in multi-line program

let { x = 2 }; 

To create a pair and get its left and right value, use a tuple:

 a = (1,2)

v1 = fst a

v2 = snd a

To zip two Lists into a List of pairs (tuples)

zip [1,2,3] ['a','b','c'] 

To define a lambda function that takes a tuple as input parameter:

(\(a,b) -> a+b) (1,2) 

Can I transpose a matrix? Yes, you can

import Data.List 

transpose [[1,2],[3,4]] 

Take length of a List

length [1,2,3] 

Take the first few from a List

take 1 [1,2,3]

Remove the first few from a List

drop 1 [1,2,3]

Drop the last one from a List

init [1,2,3,4,5]

Get the first value in a List

head [1,2,3] 

Get the last value in a List

last [1,2,3] 

Traverse a List until it doesn't match a condition

takeWhile (\x -> x > 0) [1,2,3,0,5,6] 

To get a String (List) of repeated values of a certain length

take 10 $ repeat '_' 

To define a recursive function: (you can define what a function return when a certain value is received at the parameter.)

f 0 = 0

f 1 = 0

f 2 = 1

f n = f (n-1) + f (n-2)

 To define a function f with multiple scenarios by conditions

f v

 | v > 0 = ">0"

 | v < 0 = "<0"

 | otherwise = "0"

To dynamic programming with recursion, see a tutorial on Data.Function.Memoize for fibonacci.

For example https://gist.github.com/yuhanz/e1c6793d3e8cb39fac0fa0ab9685235a


Thursday, February 10, 2022

Quick Tutorial: What is XML External Entities (XXE) Attack?

Vulnerable Scenario: your service takes in XML as input, and respond the content from the input (usually on error to indicate some parameter value).

Because XML has a DOCTYPE for variable replacement, you can easily define a variable to be replaced in the XML. For example, to define a variable myVar = "hello"

    <!DOCTYPE Query [ <!ENTITY myVar "hello" > ]>

This can be further extended to read a file on your disk for the content and assign it to the variable:

    <!DOCTYPE MySearchKeyword [ <!ENTITY myVar SYSTEM "file:///etc/passwd" > ]>

The attack: combine file reading with your XML input:

<?xml version='1.0' encoding='ISO-8859-1'?><!DOCTYPE Query [ <!ENTITY myVar SYSTEM "file:///etc/passwd" > ]> <Search>&myVar;</Search>

After our server will take the content inside <Search> to search (which is your passwd file), and it will respond with the file content to the client.


Solution:

 - Disable DTD feature in XML.

Wednesday, February 09, 2022

Quick Tutorial: What is Server-Side Request Forgery (SSRF)?

Vulnerable Scenario: when your app allows a user to send a URL to curl (or fetch, etc), potentially the user can curl a file on your server with:

curl file:///etc/passwd

So to improve the security against this, apply a check on the URL schema (not to accept with URL starting with file://)

Tuesday, January 11, 2022

FTL template - Cheatsheet

 This is a short documentation on how to use FTL template (FreeMarker Template Language)

https://freemarker.apache.org/


To try your Freemarker template online, use this interactive tool:

https://try.freemarker.apache.org/


How to check string not blank in FTL?

<#if myVariable.name?has_content>

How to null check in FTL?

<#if myVariable.name??>

How to import another FTL file?

<#import "/partials/my-partial-form.ftl" as partial>

How to call a macro in a partial form in FTL?

<@partial.myMacro myParam1 = 123/>

How to print timestamp as ISO string in FTL?

${myVariable.timestamp?datetime?string.iso}

${myVariable.timestamp?datetime?string.iso}

How to trim string in FTL?

${envelope.customer.lastName?trim}

How to keep 1 digital after decimal point in FTL?

#{y; m1}