ari/minerva
1
0
Fork 0
forked from bellwether/minerva
Code Pull requests Activity
115 commits 1 branch 0 tags 201 KiB
Commit graph

115 commits

This branch
This branch
All branches
Author SHA1 Message Date
Ari Brown
1b63d30539 renamed arg 2024-10-25 15:35:02 -04:00
Ari Brown
98355b5881 trying to reorganize how commands are done so we can get logfile grouping and tracking 2024-10-25 15:33:42 -04:00
Ari Brown
6eb31cab1e finally got streaming output working again 2024-10-18 14:23:27 -04:00
Ari Brown
95f87b2e3c updated todo; removed autoclosing of pipes (in case you supply sys.stdout); removed dumb method that gets overwritten anyways 2024-10-17 12:19:00 -04:00
Ari Brown
5e604f07f7 fixed repartition 2024-09-24 10:48:47 -04:00
Ari Brown
e3055333b3 added test files 2024-08-08 14:39:53 -04:00
Ari Brown
9f8b67b98f beginning to add web visualization of pools 2024-07-25 09:30:07 -04:00
Ari Brown
58f3d7074f fixed watch, ensured flushing of everything 2024-07-25 08:51:47 -04:00
Ari Brown
94820beccc cleaning up, adding todo 2024-07-25 08:08:47 -04:00
Ari Brown
205ed4bb18 added live output to command running 2024-07-25 08:07:56 -04:00
Ari Brown
4df9b04b2b whew, got partial reads working with named temporary files. lots of weird oddities were discovered in the process 2024-07-24 18:50:03 -04:00
Ari Brown
d67e398a69 added bare machine support (remote), and making progress with the stdout stderr redirection 2024-07-23 16:34:34 -04:00
Ari Brown
2e215c45cf using select to get partial updates of stdout and stderr from commands 2024-07-22 22:09:17 -04:00
Ari Brown
745919e587 added more examples 2024-07-03 11:04:11 -04:00
Ari Brown
a3374fd85c whoops, added self 2024-06-17 10:58:15 -04:00
Ari Brown
237c9b8c3c fixed bug in cost 2024-06-17 09:40:19 -04:00
Ari Brown
fdc0fd2ded added machine pool for easy clustering and trying to get glue support 2024-05-29 16:23:21 -04:00
Ari Brown
c2bc9e1028 added support for determining instance costs 2024-05-17 16:47:37 -04:00
Ari Brown
20ae647447 improved table deletion 2024-03-25 15:39:39 -04:00
Ari Brown
2eb404d648 version bump 2024-03-14 10:16:39 -04:00
Ari Brown
fd289257aa oops. bugfix 2024-03-14 09:13:59 -04:00
Ari Brown
e294fc062b bug fix for new sql loader 2024-03-01 13:35:06 -05:00
Ari Brown
af4f8bb999 added sql loading code from metrics calculator to minerva 2024-03-01 13:29:11 -05:00
Ari Brown
98524d3be6 fixed a bug that occurred from refactoring 2024-02-23 10:55:53 -05:00
Ari Brown
64b2cd6c00 version bump 2024-02-07 12:25:23 -05:00
Ari Brown
b453c9a515 loosened version requirements; added redshift history function; enabled Minerva to work using instance IAM role instead of explicit profile 2024-02-07 12:25:10 -05:00
Ari Brown
1e0b73eeaa updated version 2024-02-01 14:57:07 -05:00
Ari Brown
40d2eefd81 added a missing dependency; added support for redshift queries to return 0 rows 2024-02-01 14:56:48 -05:00
Ari Brown
6e249086d2 missing dependency 2024-02-01 11:04:08 -05:00
Ari Brown
e3543fe980 dropped lockfile 2024-02-01 10:45:09 -05:00
Ari Brown
0a631955c2 trying to get the gitlab runner to work 2024-01-31 16:24:02 -05:00
Ari Brown
0c00d856c0 dask clusters don't work 2024-01-31 16:21:26 -05:00
Ari Brown
138ee44609 updated athena query example 2024-01-31 16:19:24 -05:00
Ari Brown
5dccce53e9 significant improvement to the readme and verification that all the examples work 2024-01-31 16:18:32 -05:00
Ari Brown
e3c11fb1aa tidying up examples and readme 2024-01-30 18:13:56 -05:00
Ari Brown
819bf7abf3 fixed dataset aggregation; typo; and a bad empty string check 2024-01-26 11:48:35 -05:00
Ari Brown
a907be22cc trying to get the ci/cd to work 2024-01-25 14:46:25 -05:00
Ari Brown
622d104bcb update ci/cd 2024-01-25 14:43:11 -05:00
Ari Brown
14504accc7 got rid of useless print 2024-01-25 13:49:49 -05:00
Ari Brown
afd3a0b114 using autoscale gitlab runner 2024-01-25 13:43:24 -05:00
Ari Brown
ddcacdb569 fixing local paths and redshift credentials 2024-01-25 13:42:24 -05:00
Ari Brown
ae3173b510 added helpers for local files, loading templates, and an example for canceling queries 2024-01-25 11:10:50 -05:00
Ari Brown
5bd2218612 version bump 2024-01-18 12:39:06 -05:00
Ari Brown
9442c33d14 adding parallelization helpers and query cancelation 2024-01-18 12:38:42 -05:00
Ari Brown
5eb8471081 better redshift support, working on dask stuff 2023-12-21 14:23:58 -05:00
Ari Brown
e854a93e60 moving cluster scripts to new dir 2023-12-01 09:05:17 -05:00
Ari Brown
97c27f25a0 zstd w/ compression level 4 is now standard for athena query results 2023-11-30 16:48:45 -05:00
Ari Brown
210e7ebd92 improved run_cluster.py; touching up the dask test; upgrade pyarrow dependency for security patch 2023-11-28 11:12:40 -05:00
Ari Brown
c6280a3826 whoops, converting string to int 2023-11-16 13:26:26 -05:00
Ari Brown
902302c9df enhancing CLI usage of starting a cluster 2023-11-16 10:44:12 -05:00