Build data analysis and visualization tools with
BUILD DATA ANALYSIS
& VISUALIZATION TOOLS
WITH PHP
Robert Aboukhalil
@RobAboukhalil
Outline
The Basics
Backend data analysis
Show progress
Visualize results
Slides: robertaboukhalil.com/talks
Outline
The Basics
Backend data analysis
Show progress
Visualize results
Slides: robertaboukhalil.com/talks
Why web apps for data analysis?
Why web apps for data analysis?
Installing software packages can be a nightmare
If you want them to use it, it has to be usable
Access data/results from anywhere
Sample web app
ginkgo
Outline
The Basics
Backend data analysis
Show progress
Visualize results
Slides: robertaboukhalil.com/talks
Backend Data Analysis
mywebsite.com?n1=5&n2=4
9
index.php?n1=5&n2=4
<?php $n1 = $_GET['n1']; $n2 = $_GET['n2']; echo $n1 + $n2; echo shell_exec("./add.sh $n1 $n2"); ?> add.sh
#!/bin/bash (( out = $1 + $2 )); echo $out Run command line programs with PHP
// Return output as string
$allOutput = shell_exec("./analysis");
// Print output to screen + return as string
$allOutput = system("./analysis", $returnCode);
// Save output in array + return last line as string
$lastLine = exec("./analysis", $arrOutput, $returnCode);
// Print raw (binary) output to browser
header("Content-Type: image/png");
echo file_get_contents("image.png");
// Return binary output of command to browser
header("Content-Type: image/png");
passthru("./resize image.png", $returnCode);
Run programs in the background
// Run command in background
shell_exec("./analysis &");
// Run command in background (for CentOS)
$handle = popen("./analysis &", 'r');
pclose($handle);
index.php?n1=5&n2=4
<?php $n1 = $_GET['n1']; $n2 = $_GET['n2']; echo shell_exec("./add.sh $n1 $n2"); ?> add.sh
#!/bin/bash (( out = $1 + $2 )); echo $out Security
// Escape arguments
$cmd = "./analysis " . escapeshellarg($arg);
shell_exec($cmd);
// Escape command
shell_exec(escapeshellcmd($cmd));
Outline
The Basics
Backend data analysis
Show progress
Visualize results
Slides: robertaboukhalil.com/talks
Show Analysis Progress
mywebsite.com/progress.html
43%
launch.php
<?php shell_exec("./app.sh &"); ?> app.sh
#!/bin/bash for(( i = 1; i <= 100; i++)); do echo $i > progress.txt sleep 2 done status.html
<!-‐-‐ jQuery + Bootstrap.js -‐-‐> <link rel="stylesheet" href="bootstrap.min.css"> <script src="jquery.min.js"></script> <script src="bootstrap.min.js"></script> <!-‐-‐ Progress bar -‐-‐> <div class="progress"> <div id="analysis-‐progress" class="progress-‐bar"></div> </div> <script> // Set progress function getStatus() { $.get("progress.txt", function(progress) { $("#analysis-‐progress").width(progress + "%"); }); } // Do this every 1s $(document).ready(function(){ setInterval("getStatus()", 1000); }); </script> launch.php
<?php shell_exec("./app.sh &"); ?> app.sh
#!/bin/bash for(( i = 1; i <= 100; i++)); do echo $i > progress.txt sleep 2 done status.html
<!-‐-‐ jQuery + Bootstrap.js -‐-‐> <link rel="stylesheet" href="bootstrap.min.css"> <script src="jquery.min.js"></script> <script src="bootstrap.min.js"></script> <!-‐-‐ Progress bar -‐-‐> <div class="progress"> <div id="analysis-‐progress" class="progress-‐bar"></div> </div> <script> // Set progress function getStatus() { $.get("progress.txt", function(progress) { $("#analysis-‐progress").width(parseInt(progress) + "%"); }); } // Do this every 1s $(document).ready(function(){ setInterval("getStatus()", 1000); }); </script> Alternative to polling
WebSockets
Outline
The Basics
Backend data analysis
Show progress
Visualize results
Slides: robertaboukhalil.com/talks
Visualize results
mywebsite.com/results.html
launch.php
<form action="" method="POST"> x,y coordinates:<br> <textarea name="data" rows="5"></textarea><br><br> <input type="submit" name="cluster" value="Cluster"> </form> <?php if(isset($_POST['cluster'])) { $cmd = "Rscript cluster.R " . escapeshellarg($_POST['data']) . " &"; pclose(popen(escapeshellcmd($cmd), 'r')); } ?> cluster.R
# Clustering library library(mclust) # Parse input args = commandArgs(trailingOnly = TRUE) data = read.csv(con <-‐ textConnection(args[[1]]), header=FALSE) data = as.matrix(data) close(con) # Clustering allBIC = c() allClassification = list() p = 0 for(G in 2:20) { clust = Mclust(as.matrix(data), G=G) allBIC[G] = clust$bic allClassification[[G]] = clust$classification p = p + 5 writeLines(as.character(p), 'progress.txt') } # Save result to file out = c() for(i in 1:which.max(allBIC)) { tmp = data tmp[setdiff(1:dim(data)[1], which(classif == i)), 2] = "NaN" out = cbind(out, tmp[,2]) } write.csv(out, file="result.txt", quote=FALSE) status.html
<!-‐-‐ jQuery + Bootstrap.js -‐-‐> <link rel="stylesheet" href="bootstrap.min.css"> <script src="jquery.min.js"></script> <script src="bootstrap.min.js"></script> <!-‐-‐ Progress bar -‐-‐> <div class="progress"> <div id="analysis-‐progress" class="progress-‐bar"></div> </div> <script> // Set progress function getStatus() { $.get("progress.txt", function(progress) { $("#analysis-‐progress").width(parseInt(progress) + "%"); }); } // Do this every 1s $(document).ready(function(){ setInterval("getStatus()", 1000); }); </script> visualize.html
<!-‐-‐ JS Charting library -‐-‐> <script src="dygraph-‐combined.js"></script> <!-‐-‐ Where graph is plotted -‐-‐> <div id="graphdiv"></div> <!-‐-‐ Plot graph -‐-‐> <script type="text/javascript"> g = new Dygraph( $("#graphdiv"), // Div containing graph "result.txt", // Path to data file { // Other options strokeWidth: 0, pointSize: 5, drawPoints: true, } ); </script> Summary
// Run command in background
shell_exec("./analysis &");
pclose(popen("./analysis &", 'r’));
// Escape arguments
escapeshellarg($arg);
escapeshellcmd($cmd);
<!-- Bootstrap -->
<div class="progress">
<div id="analysis-progress" class="progress-bar" width="75%"></div>
</div>
<!-- JS Charting library -->
dygraphs.com
# R for data analysis
r-project.org Slides: robertaboukhalil.com/talks
© Copyright 2024