ari/minerva
1
0
Fork 0
forked from bellwether/minerva
Code Pull requests Activity
57 commits 1 branch 0 tags 201 KiB
Commit graph

57 commits

This branch
This branch
All branches
Author SHA1 Message Date
Ari Brown
937ca168ad updated dask cluster example 2023-10-12 14:59:54 -04:00
Ari Brown
b6b4b4b416 updated dask cluster example 2023-10-12 14:59:24 -04:00
Ari Brown
c2a702fb9d updated readme for another test 2023-10-12 14:58:18 -04:00
Ari Brown
2a2ff94495 updated readme to test 2023-10-12 14:55:13 -04:00
Ari Brown
c8a29cfcd3 sped up instance creation 2023-10-12 14:31:39 -04:00
Ari Brown
bfb5dda6d9 added support for distributed dataframes from athena queries 2023-10-10 21:21:22 -04:00
Ari Brown
27a1d75bb3 added monkeypatch to resolve known dask bug 2023-10-10 20:21:56 -04:00
Ari Brown
495d786103 added timing functionality for easy use 2023-10-10 19:24:16 -04:00
Ari Brown
153ab074dd trying to get ebs volume right 2023-10-10 15:42:29 -04:00
Ari Brown
9c117d6577 updated gitlab ci to always use dind 2023-10-10 14:59:42 -04:00
Ari Brown
d5532ed36e supply device name 2023-10-10 14:56:14 -04:00
Ari Brown
7c8363f5b7 version 2023-10-10 14:37:29 -04:00
Ari Brown
2caafcc344 added public/private instances and ebs block size 2023-10-10 14:37:17 -04:00
Ari Brown
5eb2e39c69 added dask clustering support 2023-10-02 17:26:02 -04:00
Ari Brown
f4dd130266 added examples 2023-09-26 18:36:41 -04:00
Ari Brown
ffdd27e506 parallel_map briefly stopped working 2023-09-26 17:53:16 -04:00
Ari Brown
02d26bbaaf changing python version reqs 2023-09-19 17:05:09 -04:00
Ari Brown
91b05578bb trying more things for ci/cd 2023-09-19 17:03:22 -04:00
Ari Brown
f2354610c2 added dependency for docker and switch gitlab ci/cd to poetry 2023-09-19 16:58:18 -04:00
Ari Brown
fe3313d83b new version for new features 2023-09-19 16:52:34 -04:00
Ari Brown
efe0f40c04 folded wharfer in and updated __init__.py 2023-09-19 16:52:20 -04:00
Ari Brown
bd3c578c06 updated version 2023-09-19 16:13:14 -04:00
Ari Brown
ffe0eddd9b output consolidation; added executable via poetry 2023-09-19 16:12:58 -04:00
Ari Brown
ab344374d9 added binary file for an easy athena console 2023-09-15 10:20:53 -04:00
Ari Brown
383185e6cb better exception info 2023-08-23 08:12:01 -04:00
Ari Brown
209f32a7de add a check to support errors being raised before tempfiles are created 2023-08-21 09:34:16 -04:00
Ari Brown
073de9fb95 version bump for parallel fix 2023-08-18 13:44:41 -04:00
Ari Brown
2613866fa3 added comments and dropped useless file 2023-08-18 12:54:45 -04:00
Ari Brown
972e6dd210 add tempfile deletion to query; ls to s3 2023-08-07 18:10:10 -04:00
Ari Brown
1d509f544c Merge branch 'add_result_config' into 'main' 
Adding ResultConfiguration for specific output location for Athena

See merge request pmo/minerva!2
 2023-08-03 18:30:19 +00:00
Roshan Punnoose
757bc74bfd Adding ResultConfiguration for specific output location for Athena 2023-08-03 14:28:12 -04:00
Ari Brown
72fd7fe005 bug fixes 2023-08-03 14:12:09 -04:00
Ari Brown
dd2edecc64 version bump 2023-08-03 13:46:24 -04:00
Ari Brown
f6c1a15b4e missing file 2023-08-03 13:46:02 -04:00
Ari Brown
6fd32f9bd3 updating version 2023-08-03 11:43:11 -04:00
Ari Brown
0e5542cecf better support for params within athena 2023-08-03 11:42:56 -04:00
Ari Brown
ea31c8e8c0 dropped inaccurate comment 2023-08-02 16:51:22 -04:00
Ari Brown
52f44a79fe parameterized queries for athena, but not for redshift. 2023-08-02 16:51:02 -04:00
Ari Brown
c4e0b71a98 slightly changed the parallel algorithm 2023-08-02 15:18:24 -04:00
Ari Brown
dc5a3519a8 added breaking changes, so bumping the version 2023-08-02 15:12:40 -04:00
Ari Brown
22746b6639 added parallel file downloading and dried up the code a touch 2023-08-02 15:12:06 -04:00
Ari Brown
68bc346e24 updating tests 2023-08-02 14:56:44 -04:00
Ari Brown
386cafe0dd dropped pycache files, renaming others 2023-08-02 14:51:47 -04:00
Ari Brown
8765e3c428 Merge branch 'various_updates' into 'main' 
Various updates to export the main classes, update dependencies, and add an...

See merge request pmo/minerva!1
 2023-08-02 14:40:02 +00:00
Roshan Punnoose
07d82c84d2 Various updates to export the main classes, update dependencies, and add an... 2023-08-02 14:40:01 +00:00
Ari Brown
2c79e43f5f updating pyproject 2023-08-01 13:59:17 -04:00
Ari Brown
c32f8359e7 restructuring as a library with athena and redshift 2023-08-01 13:19:57 -04:00
Ari Brown
adf909608d added support for redshift 2023-08-01 12:38:16 -04:00
Ari Brown
ac657f671d properly handles no results being returned 2023-07-27 15:27:48 -04:00
Ari Brown
8c498ba5dd updated readme with important information 2023-07-27 15:20:21 -04:00